Generic Support for Session and Lap in FIT files

GENERIC SUPPORT FOR PARSING INTO XDATA

.. Generically parse FIT file messages into XDATA. The current
   implementation does this for session, lap and totals messages
   but could very easily be extended to any other message type

.. Generic parsing uses metadata rather than hard coding the
   message and field types and so on

.. The FIT metadata (FITmetadata.json) has been expanded to
   include definitions of message types and all the standard
   fields within the message types

.. The existing hard-coded parsing remains to extract data
   and apply directly to ridefile samples and metadata. The
   generic parser simply adds additional tabs on the data
   view as XDATA so users can access it.

   CODE REFACTORING, COMMENTS AND BUG FIXES

.. At some point the code needs to be refactored as it is
   janky and needs to align with the rest of the codebase

.. Includes a mild refactor renaming some of the classes/structs
   and variables to reflect what they actually are, for example:

       FitFileReadState -> FitFileParser
       FitDefinition -> FitMessage

.. Added lots of code comments and re-organised the code
   into clear sections to help navigate what is a very
   cumbersome source file, this breaks git blame history
   but is worth the loss (you can checkout an earlier commit
   to do a full blame)

.. Changed debugging levels to be more helpful

.. Generally I did not change any code, but there were a
   couple of serious bugs that needed to be corrected:

      Field definitions gets the type wrong in a couple of
      places since the type is stored in the low 4 bits:
      type = value & 0x1F

      The decodeDeveloperFieldDescription function did not
      check for NA_VALUEs for scale, offset, native field

.. For less serious bugs I added FIXME comments throughout the code

Fixes #4416

util/fit/Makefile

@@ -1,3 +1,3 @@
-../../src/Resources/json/FITmetadata.json: fit_example.h nongarmin.json fitprod.py
+../../src/Resources/json/FITmetadata.json: fit_example.h nongarmin.json fields.json fitprod.py
 	python3 fitprod.py > ../../src/Resources/json/FITmetadata.json
 

util/fit/README

@@ -3,12 +3,36 @@ FITmetadata.json generator
 
 For Garmin device updates:
 
-    Make sure you download the latest FIT SDK from thisisant.com and extract
-    the c/fit_example.h file and place it in this directory.
+    Make sure you download the latest FIT SDK from https://developer.garmin.com/fit/download/
+    and extract the c/fit_example.h file and place it in this directory.
 
     Do not edit this file, we expect it to be lifted directly from the SDK
     and it will be replaced as new versions of the SDK are released
 
+For Garmin profile updates:
+
+   The fields.json file in this directory is metadata describing most of the standard
+   fields and is based upon the Profile.xls file in the SDK. If new fields are or profiles
+   are added and you want them to be parsed via metadata then update fields.json to
+   include the entries.
+
+   At present the FIT ride file reader code is a mix of hard coded parsing for
+   the majority of the message types, but the code for parsing session and lap
+   message types does use the metadata to control parsing (and puts the results
+   into the XDATA section of a ridefile)
+
+   If you add new profile then you should use metadata to drive the parsing and
+   add the data to the XDATA section to avoid impacting existing code.
+
+Generate a new FITmetadata.json file
+
+    There is a Makefile in this directory that will refresh baed upon the timestamp
+    of the fit_example.h. To refresh simply run 'make'. If you wish to re-run and
+    regenerate 'touch fit_example.h; make' will do that.
+
+    You will need python3 installed since the generator is a python script. Please
+    do not edit this script but raise a github issue if you find it is broken.
+
 When Garmin devices are not yet supported by the SDK
 
     The SDK lags behind device availability in this case you should add

util/fit/fitprod.py

@@ -5,13 +5,17 @@ print('{\n    "VERSION":' + str(int(time.time())) + ',')
 print('    "COMMENT":"Do not edit this file directly it is generated.",')
 print('    "PRODUCTS":[')
 
+#
 # output the known products captured by GoldenCheetah users
+#
 nongarmin = open("nongarmin.json", "r")
 lines = nongarmin.readlines()
 for line in lines:
     print("        " + line, end="")
 
+#
 # output garmin products as described in the FIT SDK
+#
 sdkheader = open("fit_example.h","r")
 lines = sdkheader.readlines()
 pre="        "
@@ -26,8 +30,9 @@ for line in lines:
             pre=",\n        "
 print("\n    ],\n")
 
+#
 # manufacturers list from FIT SDK
-
+#
 print('    "MANUFACTURERS":[')
 pre="        "
 for line in lines:
@@ -36,5 +41,29 @@ for line in lines:
         print(pre+ '{ "manu":' + match.group(2).strip() + ', "name":"' +  match.group(1).strip().replace('_',' ').title() + '" }', end="")
         pre=",\n        "
 
-print("\n    ]\n}")
+print("\n    ],\n")
 
+#
+# field numbers
+#
+fields = open("fields.json", "r")
+fieldlines = fields.readlines()
+for line in fieldlines:
+    print(line, end="")
+
+#
+# Message number description
+#
+# // #define FIT_MESG_NUM_HR_ZONE                                                     ((FIT_MESG_NUM)8)
+
+print('    "MESSAGES":[')
+pre="        "
+for line in lines:
+    match = re.search("FIT_MESG_NUM_([^ \t]*).*\(\(FIT_MESG_NUM\)([ 0-9]*)", line)
+    if match:
+        num = int(match.group(2).strip())
+        if num > 0:
+            print(pre+ '{ "num":', num, ', "desc":"' +  match.group(1).strip().replace('_',' ').title() + '" }', end="")
+            pre=",\n        "
+
+print("\n    ]\n}")