added ability to define multiple line definitions

[eddb7]

Added multiple line functionality
All Line defintions must start as so even if defining a singular line pattern

lines:

• start:
  end:
  line:
• start:
  end:
  line:

Without the - it will error. My yaml parsing knowledge is pretty basic so maybe someone could fix this so it behaves as the tables plugin does ( no need for - for single case)

[adrienjacob]

Would be very helpful indeed !

[m3nu]

Could you sign off on this, @hbrunn ? I don't know enough about the line plugin. Thanks!

[adrienjacob]

I'm testing your PR... For now I see a slight issue : the line row[name] = self.coerce_type(row[name], types[name]) uses only the last declared "types" block... Eg:

- start:
  end:
  line:
  types:
      date_something: date
      amount_something: float
- start:
  end:
  line:
  types:
      amount_something_else: float

Only amount_something_else would be converted to float... To solve this, one has basically to declare all types in the last block only.

[adrienjacob]

Other suggestion here:

if not start or not end:
  logger.warning('no lines found - start %s, end %s', start, end)
  return

I would rather put a continue than a return, as other line blocks may still be useful.

[dseibel]

I stumbled on this and it looks usefull for something I would like to do. @adrienjacob I tried the issue you mentioned about types but don't see the same behaviour.

I have this in my yml file:

`lines:

• start: Start meter reading (s)\s+End meter reading (s)\s+Conv. factor\s+Usage (GJ)\s+Meas. dmd.\s+Billed demand
  end: '\s+Retailer charges -'
  line: '\s+(?P<gas_start_date>\w{3}.\d+/\d+)\s+(?P<gas_start_reading>\d+).+(?P<gas_end_date>\w{3}.\d+/\d+)\s+(?P<gas_end_reading>\d+).+(?P\d+.\d+)'
  types:
  gas_start_date: date
  gas_end_date: date
• start: Floating price natural gas charges
  end: 'Site ID:'
  line: '(?P<gas_usage>\d+.\d+).+$(?P<gas_entry_rate>\d+.\d+)/GJ =\s+(?P<gas_entry_total>\d+.\d+)'
  types:
  gas_entry_rate: int`

and the output of lines is:

lines [{'gas_start_date': datetime.datetime(2019, 10, 22, 0, 0), 'gas_start_reading': '391', 'gas_end_date': datetime.datetime(2019, 10, 31, 0, 0), 'gas_end_reading': '394', 'gasUsage': '3.16'}, {'gas_start_date': datetime.datetime(2019, 11, 1, 0, 0), 'gas_start_reading': '394', 'gas_end_date': datetime.datetime(2019, 11, 19, 0, 0), 'gas_end_reading': '401', 'gasUsage': '6.86'}, {'gas_usage': '3.16', **'gas_entry_rate': 3, 'gas_entry_total': '11.29'}, {'gas_usage': '6.86', 'gas_entry_rate': 3, 'gas_entry_total': '24.69'}]

and if I remove the types config the output is:

`lines [{'gas_start_date': 'Oct 22/19', 'gas_start_reading': '391', 'gas_end_date': 'Oct 31/19', 'gas_end_reading': '394', 'gasUsage': '3.16'}, {'gas_start_date': 'Nov 1/19', 'gas_start_reading': '394', 'gas_end_date': 'Nov 19/19', 'gas_end_reading': '401', 'gasUsage': '6.86'}, {'gas_usage': '3.16', 'gas_entry_rate': '3.57239', 'gas_entry_total': '11.29'}, {'gas_usage': '6.86', 'gas_entry_rate': '3.59884', 'gas_entry_total': '24.69'}]

Your other suggestion of putting a continue than a return seems to work well.

[bosd]

Does this one offer imilar functionality of #308 ?

[bosd]

EDIT: I think this old PR can be closed, as it has become a parser now.

FYI, I am working on an implementation to forward migrate this idea/code.

[rmilecki]

Superseded by #378