Merge pull request #176 from kellyjonbrazil/dev

Dev v1.17.1

.github/workflows/pythonapp.yml

@@ -14,7 +14,7 @@ jobs:
     strategy:
       matrix:
         os: [macos-latest, ubuntu-latest, windows-latest]
-        python-version: [3.6, 3.7, 3.8, 3.9]
+        python-version: [3.6, 3.7, 3.8, 3.9, 3.10.0]
 
     steps:
     - uses: actions/checkout@v2

CHANGELOG

@@ -1,5 +1,13 @@
 jc changelog
 
+20211030 v1.17.1
+- Fix file parser for gzip files
+- Fix uname parser for cases where the 'processor' and/or 'hardware_platform' fields are missing on linux
+- Fix uname parser on FreeBSD
+- Add lsusb parser tested on linux
+- Add CSV file streaming parser
+- Add testing for Python 3.10.0
+
 20210923 v1.17.0
 - Note to Package Maintainers: please see note at 20210720 v1.16.0
 - Add wrapping of warning and error messages

EXAMPLES.md

@@ -1917,6 +1917,128 @@ lsof | jc --lsof -p          # or:  jc -p lsof
   }
 ]
 ```
+### lsusb
+```bash
+lsusb -v | jc --lsusb -p          # or: jc -p lsusb -v
+```
+```json
+[
+  {
+    "bus": "002",
+    "device": "001",
+    "id": "1d6b:0001",
+    "description": "Linux Foundation 1.1 root hub",
+    "device_descriptor": {
+      "bLength": {
+        "value": "18"
+      },
+      "bDescriptorType": {
+        "value": "1"
+      },
+      "bcdUSB": {
+        "value": "1.10"
+      },
+      ...
+      "bNumConfigurations": {
+        "value": "1"
+      },
+      "configuration_descriptor": {
+        "bLength": {
+          "value": "9"
+        },
+        ...
+        "iConfiguration": {
+          "value": "0"
+        },
+        "bmAttributes": {
+          "value": "0xe0",
+          "attributes": [
+            "Self Powered",
+            "Remote Wakeup"
+          ]
+        },
+        "MaxPower": {
+          "description": "0mA"
+        },
+        "interface_descriptors": [
+          {
+            "bLength": {
+              "value": "9"
+            },
+            ...
+            "bInterfaceProtocol": {
+              "value": "0",
+              "description": "Full speed (or root) hub"
+            },
+            "iInterface": {
+              "value": "0"
+            },
+            "endpoint_descriptors": [
+              {
+                "bLength": {
+                  "value": "7"
+                },
+                ...
+                "bmAttributes": {
+                  "value": "3",
+                  "attributes": [
+                    "Transfer Type  Interrupt",
+                    "Synch Type  None",
+                    "Usage Type  Data"
+                  ]
+                },
+                "wMaxPacketSize": {
+                  "value": "0x0002",
+                  "description": "1x 2 bytes"
+                },
+                "bInterval": {
+                  "value": "255"
+                }
+              }
+            ]
+          }
+        ]
+      }
+    },
+    "hub_descriptor": {
+      "bLength": {
+        "value": "9"
+      },
+      ...
+      "wHubCharacteristic": {
+        "value": "0x000a",
+        "attributes": [
+          "No power switching (usb 1.0)",
+          "Per-port overcurrent protection"
+        ]
+      },
+      ...
+      "hub_port_status": {
+        "Port 1": {
+          "value": "0000.0103",
+          "attributes": [
+            "power",
+            "enable",
+            "connect"
+          ]
+        },
+        "Port 2": {
+          "value": "0000.0103",
+          "attributes": [
+            "power",
+            "enable",
+            "connect"
+          ]
+        }
+      }
+    },
+    "device_status": {
+      "value": "0x0001",
+      "description": "Self Powered"
+    }
+  }
+]
+```
 ### mount
 ```bash
 mount | jc --mount -p          # or:  jc -p mount

README.md

@@ -115,6 +115,7 @@ The JSON output can be compact (default) or pretty formatted with the `-p` optio
 - `--crontab` enables the `crontab` command and file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab))
 - `--crontab-u` enables the `crontab` file parser with user support ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u))
 - `--csv` enables the CSV file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv))
+- `--csv-s` enables the CSV file streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s))
 - `--date` enables the `date` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/date))
 - `--df` enables the `df` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/df))
 - `--dig` enables the `dig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dig))
@@ -147,6 +148,7 @@ The JSON output can be compact (default) or pretty formatted with the `-p` optio
 - `--lsblk` enables the `lsblk` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk))
 - `--lsmod` enables the `lsmod` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod))
 - `--lsof` enables the `lsof` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof))
+- `--lsusb` enables the `lsusb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb))
 - `--mount` enables the `mount` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mount))
 - `--netstat` enables the `netstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat))
 - `--ntpq` enables the `ntpq -p` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq))

docs/parsers/csv_s.md

@@ -0,0 +1,77 @@
+[Home](https://kellyjonbrazil.github.io/jc/)
+
+# jc.parsers.csv_s
+jc - JSON CLI output utility `csv` file streaming parser
+
+> This streaming parser outputs JSON Lines
+
+The `csv` streaming parser will attempt to automatically detect the delimiter character. If the delimiter cannot be detected it will default to comma. The first row of the file must be a header row.
+
+Note: The first 100 rows are read into memory to enable delimiter detection, then the rest of the rows are loaded lazily.
+
+Usage (cli):
+
+    $ cat file.csv | jc --csv-s
+
+Usage (module):
+
+    import jc.parsers.csv_s
+    result = jc.parsers.csv_s.parse(csv_output)
+
+Schema:
+
+    csv file converted to a Dictionary: https://docs.python.org/3/library/csv.html
+
+    {
+      "column_name1":     string,
+      "column_name2":     string
+    }
+
+Examples:
+
+    $ cat homes.csv
+    "Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres", "Taxes"
+    142, 160, 28, 10, 5, 3,  60, 0.28,  3167
+    175, 180, 18,  8, 4, 1,  12, 0.43,  4033
+    129, 132, 13,  6, 3, 1,  41, 0.33,  1471
+    ...
+
+    $ cat homes.csv | jc --csv-s
+    {"Sell":"142","List":"160","Living":"28","Rooms":"10","Beds":"5","Baths":"3","Age":"60","Acres":"0.28","Taxes":"3167"}
+    {"Sell":"175","List":"180","Living":"18","Rooms":"8","Beds":"4","Baths":"1","Age":"12","Acres":"0.43","Taxes":"4033"}
+    {"Sell":"129","List":"132","Living":"13","Rooms":"6","Beds":"3","Baths":"1","Age":"41","Acres":"0.33","Taxes":"1471"}
+    ...
+
+
+## info
+```python
+info()
+```
+Provides parser metadata (version, author, etc.)
+
+## parse
+```python
+parse(data, raw=False, quiet=False, ignore_exceptions=False)
+```
+
+Main text parsing generator function. Returns an iterator object.
+
+Parameters:
+
+    data:              (iterable)  line-based text data to parse (e.g. sys.stdin or str.splitlines())
+    raw:               (boolean)   output preprocessed JSON if True
+    quiet:             (boolean)   suppress warning messages if True
+    ignore_exceptions: (boolean)   ignore parsing exceptions if True
+
+Yields:
+
+    Dictionary. Raw or processed structured data.
+
+Returns:
+
+    Iterator object
+
+## Parser Information
+Compatibility:  linux, darwin, cygwin, win32, aix, freebsd
+
+Version 1.0 by Kelly Brazil ([email protected])

docs/parsers/file.md

@@ -87,4 +87,4 @@ Returns:
 ## Parser Information
 Compatibility:  linux, aix, freebsd, darwin
 
-Version 1.3 by Kelly Brazil ([email protected])
+Version 1.4 by Kelly Brazil ([email protected])