Option to change DataStream parameters during .submit_data()?

[ehpor]

For many services, we want to just submit data, and change DataStream shape and dtype to whatever we are submitting. It would be nice if there is an option in DataStream.submit_data() for that. Something like:

self.images.submit_data(image, match_datastream_parameters=True)

Opinions? @raphaelpclt @lanemeier7

[raphaelpclt]

Hmmm I am a bit surprised. That means all the process / experiments / services reading from this datastream must be ready to adapt to a type change?
Do you have a case in mind where you'd want to change type on the fly? For the shape, it's already possible to submit a smaller frame right?

[lanemeier7]

Yes, modification of the original data stream size was needed for the accufiz service. It looks like historically the data was transformed and clipped a bit based radius mask resulting a change in the initial data stream which was set by sensor size in the .yml. Or you could imagine a camera with a windowing capability that can be changed on the fly

[ehpor]

@raphaelpclt I specifically mean, have an argument in .submit_data() that does the following:

catkit2/catkit2/services/zwo_camera/zwo_camera.py

Lines 189 to 193 in f90c03f

	# Make sure the data stream has the right size and datatype.
	has_correct_parameters = np.allclose(self.images.shape, [self.height, self.width])
	if not has_correct_parameters:
	self.images.update_parameters('float32', [self.height, self.width], self.NUM_FRAMES_IN_BUFFER)

and also for the dtype.

We've been repeating this piece of code a few times now, and while I originally thought being strict was the way to go, having an opt-in which does this for you, could be useful.

[ehpor]

@lanemeier7 Indeed we do this for windowing cameras, dOTF intermediate results, etc... Basically anything where the shape is not known in advance and can change on the fly. You still pay a runtime cost when switching dimensions, but unless you're doing that every single frame (which you should not be doing), then it's fine.

[ehpor]

Or maybe make DataStream::UpdateParameters() bail out quickly when no parameter changes are needed?

Or maybe have a DataStream.match_parameters(data) function that matches the data stream to the shape and dtype of the data? But honestly, I'd prefer a flag in DataStream.submit_data() if we're going that route.

[lanemeier7]

The flag sounds the cleanest to me, then it won't add any lines of code to the service logic.