EMU got serious about metadata

[atruskie]

This is a huge release!

The metadata command

The metadata command has been enhanced to show:

• more detailed metadata from SM4s and SM3s
  • including total gain used to create a recording (Extracting sensitivity and gain for SM4 #331)
  • and high precision timers sometimes found in SM4s with GPS modules attached (Sub-second precision in SM4 #330)
  • the sensitivity field, if present is now also set
• metadata for AudioMoths inspired by the metamoth project (Feature Request: Add support for AudioMoth metadata extraction #349)
• the make and model of more sensors are now reported (Metadata: fill vendor and sensor name fields #296)
• we've added some new fields to the metadata format, including:
  • RecordingStatus (currently only populated for AudioMoths)
  • Notices show showing warnings and errors (currently rarely used)
  • Sensor.FirmwareName for showing the names of custom firmwares (currently only used for AudioMoths)

There are however some breaking changes to the metadata format too:

• SourcePath has been renamed to Path for simplicity. The extra context is not needed as every result only ever refers to one path.
• Sensor.Vendor has been renamed to Sensor.Make for consistency with with upcoming planned support for the GUANO metadata format
• Sensor.Temperature has been clarified to mean the onboard/internal temperature of a sensor
• Sensor.Microphone.Channel has been changed so that is uses standard channel index descriptions. This means the left/mono channel is now consistently 0 whereas it previously reported as 1. The same "minus one" change has been applied to other channels too.
• Some locations reported for SM4/SM3s were previously reported incorrectly as being the wrong side of the prime meridian. This has been fixed.

Check out our updates metadata command docs for more details.

The metadata dump command

There is a wealth of information in the varying types of metadata provided by different sensors. The metadata command is great for exposing this metadata in a consistent format, but at the cost of losing some of the detail. The metadata dump (#355) command is a new command that dumps the low-level, detailed information parsed from the headers of audio recordings, with minimal processing.

For example: did you know the entire schedule used to program a SM3/SM4 is stored in the header of every recording? The metadata dump command can show you this information!

Here is the metadata dump output for a SM3 recording:

You can see the settings a sensor was programmed with, it's schedule, and most of the WAMD metadata block decoded in a easy to read format.

metadata dump is supported for:

• AudioMoth files
• SM3/SM4 files
• Frontier Labs BAR-LT files

The information is more detailed, but varies among the different vendors. It's not uniform. Because of the lack of uniformity we can not emit CSV output for the metadata dump command.

There are more examples in our metadata command docs.

Other changes

• We've added a metadata show command as an alias for the plain metadata command to make both metadata show and metadata dump commands more consistent.
• We've added a new colorful and better laid out default formatter which makes viewing metadata in the console easier
• Considerable effort went into reverse engineering the WAMD metadata format. The documentation for this format can be found in the schedule structure document. Let me know if this document is useful to you!
• Makes FL011 fix more resiliant by @atruskie in Makes FL011 fix more resiliant #360
• Rename io handle bug by @atruskie in Rename io handle bug #364

Final notes

EMU is already a valuable tool for us but it is hardly finished. Please let us know if there is a feature you would like to see added.

We're keen to add support for more sensors, and we're also keen to add support for the GUANO metadata format. If you have any experience with GUANO and would like to help us add support for it, please get in touch!

Full Changelog: 6.5.0...7.0.0

---

This discussion was created from the release EMU got serious about metadata.