The mDOS parser is a tool to read and convert the survey data taken with one of our MS-xxx soil/sediment mapping sensors. The use of the tool is very straightforward: (1) select the data you've downloaded from your sensor; (2) select a template for parsing and (3) run. the parser by selecting the PARSE button.
The tool has 3 pages:
- The FILES page. This page contains a list of the files you have selected for parsing;
- The TEMPLATES page. This page allows to create or select a template. Templates allow to create a selection of the data fields to include in the output file(s)
- The INFO page. This page contains a bit of info on the software
The images below show these pages (click to enlarge). In the following the functions of the tool will be described in more detail.
Download the installer here: Setup_Parser.exe
Data files and templates
The data files produced by the Medusa Soil Mapping Systems (SMS) are stored in json format. JSON is a structured and human-readable data formatting system that is used widely in modern data acquisition systems. The JSON files consist of a header part that contains info on the sensor at hand, and on the data it is capable of gathering. The rest of the file contains the data resulting from your survey work. Each line is an individual record containing the data taken at a certain moment in time. This data can be from the embedded GPS, from the embedded pressure, temp and humidity sensors, or from external instruments. And the data contains the output of the gamma-ray spectrometer which is at the heart of your SMS.
However, the output of a Medusa system can be pretty verbose and may contain all kinds of info you not need. You can use a template to set what data you want to see in the output and what data can be omitted. It is important to note that you will always need to select a template - parsing cannot proceed without using a template. And if you have not yet created a template, then you should first create one.
To create a template, open the TEMPLATES page and select the CREATE button. You now get the form as shown here:
The screenshot shown here shows the situation after the file listed in the edit box on top of the form has been read (by selecting the READ button). The left box contains the data types that are generated by your SMS; the right-hand box shows information on the selected field (in this case, it shows the information of the data type LAT which is part of the GPS dataset). By (de)selecting different fields of the GPS object, you can govern what data gets in your output files. By (de)selecting the complete object (like shown here for the SPECTRO_2657 and TaggedSpectrum objects), you omit the data from this object all-together.
Another important selection to make, is in teh box labeled "Select spectrum field to parse" on the lower-left of the form. In this box you can select the data field that the parser will use as a synchronisation field. Normally you will select either a field labeled STABSPECTRO_xxx or SPECTRO_xxx as these objects contain the gamma-ray spectra taken with your device.
Once you are done with making the selection, you can store the template by selecting the STORE button:
SPECTRO or STABSPECTRO?
The latest versions of the medusa Detector Operating System (mDOS) store spectra both in a "raw" format (as 512 channels) and in an energy-stabilised format. In the latter, each spectrum is shifted in such a way that all peaks arrive at the proper gamma-ray energy. This process is called "energy-stabilisation" and used to be a procedure that could only be done in post-processing (using Gamman). Now, you can choose whether to use the "raw" spectra, or the energy-stabilised ones. Normally you would use the stabilised spectra by choosing the STABSPEC_xxx field in the parser.
Getting ready to parse
The steps you need to take to parse data are as follows:
- Select data files. You can also drag the files to parse from the Windows Explorer, and drop them on the Parser;
- Select (or check) the template you want to use. Note: the last used template is stored when you close the program and will be listed as the active template when you run the software again;
- Choose which file types to export
- Gamman → this will produce a GSF (Gamman Survey File) that can be used right away in Gamman;
- CSV → this will create a comma-separated file containing all aux fields you've selected in your template, plus the gamma-ray spectra.
- Click PARSE
Reading and sorting the data
The first step of the parser is to read all datafiles and create a list in memory of all data it found. This list is then sorted on time. This can be a lengthy process, depending on the amount of data you have selected for processing
Synchronisation of data
The second task of the parser is to synchronise all "auxiliary" data to the gamma-ray spectra measured by the spectrometer. This synchronisation is needed since all data is logged at its own pace. For instance, it can be your GPS is set to record positions every 0,1 second. The spectra are normally taken once per second, so the GPS positions are not necessarily taken at exactly the same moment in time as the spectrometer. The synchroniser basically runs through the list of spectra, and for each spectrum tries to find the aux data nearest in time. It then interpolates these values to the timestamp of the current spectrum.
After the synchronisation has finished, the data is stored to a GSF or CSV file, as mentioned above, and you are ready for the next steps in your data processing.