-
Notifications
You must be signed in to change notification settings - Fork 120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implemented microphone modes #853
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,104 +1,178 @@ | ||
| from dragonfly import get_engine, get_current_engine | ||
| from castervoice.lib import settings, printer | ||
| from dragonfly import get_engine, get_current_engine, FuncContext, Function, MappingRule, Grammar, Choice, Dictation | ||
| from castervoice.lib import printer | ||
|
|
||
| engine = get_current_engine().name | ||
| if engine == 'natlink': | ||
| import natlink | ||
|
|
||
| # TODO: Implement a grammar exclusivity for non-DNS engines in a separate class | ||
|
|
||
| class EngineModesManager(object): | ||
| """ | ||
| Manages engine modes and microphone states using backend engine API and through dragonfly grammar exclusivity. | ||
| """ | ||
| engine = get_current_engine().name | ||
| if engine == 'natlink': | ||
| import natlink | ||
|
|
||
| engine_modes = {"normal":0, "command":2, "dictation":1,"numbers":3, "spell":4} | ||
| mic_modes = ["on", "sleeping", "off"] | ||
| engine_modes = {"normal": 0, "command": 2, | ||
| "dictation": 1, "numbers": 3, "spell": 4} | ||
| mic_modes = {"on": 5, "sleeping": 6, "off": 7} | ||
| engine_state = None | ||
| previous_engine_state = None | ||
| mic_state = None | ||
| timer_callback = None | ||
|
|
||
| def initialize(self): | ||
| @classmethod | ||
| def initialize(cls): | ||
| # Remove "normal" and "off" from 'states' for non-DNS based engines. | ||
| if self.engine != 'natlink': | ||
| self.engine_modes.pop("normal", 0) | ||
| self.mic_modes.remove("off") | ||
| if engine != 'natlink': | ||
| cls.engine_modes.pop("normal", 0) | ||
| cls.mic_modes.pop("off", 7) | ||
| #if self.engine == 'natlink': | ||
| # Sets 1st index key ("normal" or "command") depending on engine type as default mode | ||
| self.engine_state = self.previous_engine_state = next(iter(self.engine_modes.keys())) | ||
|
|
||
| cls.engine_state = cls.previous_engine_state = next( | ||
| iter(cls.engine_modes.keys())) | ||
| # Timer to synchronize natlink.getMicState/SetRecognitionMode with mode_state in case of changed by end-user via DNS GUI. | ||
| if engine == 'natlink' and cls.timer_callback is None: | ||
| cls.timer_callback = get_current_engine().create_timer(callback=cls._sync_mode, interval=1) | ||
| cls.timer_callback.start() | ||
|
|
||
| def set_mic_mode(self, mode): | ||
| @classmethod | ||
| def set_mic_mode(cls, mode): | ||
| """ | ||
| Changes the engine microphone mode | ||
| 'on': mic is on | ||
| 'sleeping': mic from the sleeping and can be woken up by command | ||
| 'off': mic off and cannot be turned back on by voice. (DNS Only) | ||
| :param mode: str | ||
| 'on': mic is on | ||
| 'sleeping': mic from the sleeping and can be woken up by command | ||
| 'off': mic off and cannot be turned back on by voice. (DPI Only) | ||
| """ | ||
| if mode in self.mic_modes: | ||
| self.mic_state = mode | ||
| if self.engine == 'natlink': | ||
| self.natlink.setMicState(mode) | ||
| # From here other engines use grammar exclusivity to re-create the sleep mode | ||
| #if mode != "off": # off does not need grammar exclusivity | ||
| #pass | ||
| # TODO: Implement mic mode sleep mode using grammar exclusivity. This should override DNS is built in sleep grammar but kept in sync automatically with natlink.setmic_state | ||
| else: | ||
| printer.out("Caster: 'set_mic_mode' is not implemented for '{}'".format(self.engine)) | ||
| if mode in cls.mic_modes: | ||
| cls.mic_state = mode | ||
| if engine == 'natlink': | ||
| natlink.setMicState(mode) | ||
| # Overrides DNS/DPI is built in sleep grammar | ||
| ExclusiveManager(mode, modetype="mic_mode") | ||
| else: | ||
| printer.out("Caster: '{}' is not a valid. set_mic_state modes are: 'off' - DNS Only, 'on', 'sleeping'".format(mode)) | ||
| printer.out( | ||
| "Caster: '{}' is not valid. set_mic_state modes are: 'off' - DPI Only, 'on', 'sleeping'".format(mode)) | ||
|
|
||
|
|
||
| def get_mic_mode(self): | ||
| @classmethod | ||
| def get_mic_mode(cls): | ||
| """ | ||
| Returns mic state. | ||
| mode: string | ||
| mode: str | ||
| """ | ||
| return self.mic_state | ||
|
|
||
| return cls.mic_state | ||
|
|
||
| def set_engine_mode(self, mode=None, state=True): | ||
| @classmethod | ||
| def set_engine_mode(cls, mode=None, state=True): | ||
| """ | ||
| Sets the engine mode so that only certain types of commands/dictation are recognized. | ||
| 'state': Bool - enable/disable mode. | ||
| :param state: Bool - enable/disable mode. | ||
| 'True': replaces current mode (Default) | ||
| 'False': restores previous mode | ||
| 'normal': dictation and command (Default: DNS only) | ||
| 'dictation': Dictation only | ||
| 'command': Commands only (Default: Other engines) | ||
| 'numbers': Numbers only | ||
| 'spell': Spelling only | ||
| :param mode: str | ||
| 'normal': dictation and command (Default: DPI only) | ||
| 'dictation': Dictation only | ||
| 'command': Commands only (Default: Other engines) | ||
| 'numbers': Numbers only | ||
| 'spell': Spelling only | ||
| """ | ||
| if state and mode is not None: | ||
| # Track previous engine state | ||
| # TODO: Timer to synchronize natlink.getMicState() with mengine_state in case of changed by end-user via DNS GUI. | ||
| self.previous_engine_state = self.engine_state | ||
| cls.previous_engine_state = cls.engine_state | ||
| else: | ||
| if not state: | ||
| # Restore previous mode | ||
| mode = self.previous_engine_state | ||
| mode = cls.previous_engine_state | ||
| else: | ||
| printer.out("Caster: set_engine_mode: 'State' cannot be 'True' with a undefined a 'mode'") | ||
|
|
||
| if mode in self.engine_modes: | ||
| if self.engine == 'natlink': | ||
| printer.out( | ||
| "Caster: set_engine_mode: 'State' cannot be 'True' with a undefined a 'mode'") | ||
|
|
||
| if mode in cls.engine_modes: | ||
| if engine == 'natlink': | ||
| try: | ||
| self.natlink.execScript("SetRecognitionMode {}".format(self.engine_modes[mode])) # engine_modes[mode] is an integer | ||
| self.engine_state = mode | ||
| natlink.execScript("SetRecognitionMode {}".format( | ||
| cls.engine_modes[mode])) # engine_modes[mode] is an integer | ||
| cls.engine_state = mode | ||
| ExclusiveManager(mode, modetype="engine_mode") | ||
| except Exception as e: | ||
| printer.out("natlink.execScript failed \n {}".format(e)) | ||
| else: | ||
| # TODO: Implement mode exclusivity. This should override DNS is built in sleep grammar but kept in sync automatically with natlinks SetRecognitionMode | ||
| # Once DNS enters its native mode exclusivity will override the any native DNS mode except for normal/command mode. | ||
| if self.engine == 'text': | ||
| self.engine_state = mode | ||
| # TODO: Implement set_engine_mode exclusivity. This should override DPI is built mode but kept in sync automatically. | ||
| if engine == 'text': | ||
| cls.engine_state = mode | ||
| else: | ||
| printer.out("Caster: 'set_engine_mode' is not implemented for '{}'".format(self.engine)) | ||
| printer.out( | ||
| "Caster: 'set_engine_mode' is not implemented for '{}'".format(engine)) | ||
| else: | ||
| printer.out("Caster: '{}' mode is not a valid. set_engine_mode: Modes: 'normal'- DNS Only, 'dictation', 'command', 'numbers', 'spell'".format(mode)) | ||
|
|
||
| printer.out( | ||
| "Caster: '{}' mode is not valid. set_engine_mode: Modes: 'normal'- DPI Only, 'dictation', 'command', 'numbers', 'spell'".format(mode)) | ||
|
|
||
| def get_engine_mode(self): | ||
| @classmethod | ||
| def get_engine_mode(cls): | ||
| """ | ||
| Returns engine mode. | ||
| mode: str | ||
| mode: str | ||
| """ | ||
| return cls.engine_state | ||
|
|
||
| @classmethod | ||
| def _sync_mode(cls): | ||
| """ | ||
| Synchronizes Caster exclusivity modes an with DNS/DPI GUI built-in modes state. | ||
| """ | ||
| return self.engine_state | ||
| # TODO: Implement set_engine_mode logic with modes not just mic_state. | ||
| mic_state = cls.get_mic_mode() | ||
| if mic_state is None: | ||
| cls.mic_state = natlink.getMicState() | ||
| else: | ||
| if natlink.getMicState() != mic_state: | ||
| cls.set_mic_mode(natlink.getMicState()) | ||
|
|
||
|
|
||
| class ExclusiveManager: | ||
| """ | ||
| Loads and switches exclusivity for caster modes | ||
| :param mode: str | ||
| :param modetype: 'mic_mode' or 'engine_mode' str | ||
| """ | ||
| # TODO: Implement set_engine_mode exclusivity with mode rules. | ||
| # TODO: Implement timer for sleep mode. | ||
| # TODO: Implement hotkey for microphone on-off | ||
| sleep_grammar = None | ||
| sleeping = False | ||
|
|
||
| sleep_rule = MappingRule( | ||
| name="sleep_rule", | ||
| mapping={ | ||
| "caster <mic_mode>": Function(lambda mic_mode: EngineModesManager.set_mic_mode(mode=mic_mode)), | ||
| "<text>": Function(lambda text: False) | ||
| }, | ||
| extras=[Choice("mic_mode", { | ||
| "off": "off", | ||
| "on": "on", | ||
| "sleep": "sleeping", | ||
| }), | ||
| Dictation("text")], | ||
| context=FuncContext(lambda: ExclusiveManager.sleeping), | ||
| ) | ||
|
|
||
| def __init__(self, mode, modetype): | ||
| if modetype == "mic_mode": | ||
| if not isinstance(ExclusiveManager.sleep_grammar, Grammar): | ||
| grammar = ExclusiveManager.sleep_grammar = Grammar("sleeping") | ||
| grammar.add_rule(self.sleep_rule) | ||
| grammar.load() | ||
| if mode == "sleeping": | ||
| self.set_exclusive(state=True) | ||
| printer.out("Caster: Microphone is sleeping") | ||
| if mode == "on": | ||
| self.set_exclusive(state=False) | ||
| printer.out("Caster: Microphone is on") | ||
| if mode == "off": | ||
| printer.out("Caster: Microphone is off") | ||
| else: | ||
| printer.out("{}, {} not implemented".format(mode, modetype)) | ||
|
|
||
| def set_exclusive(self, state): | ||
| grammar = ExclusiveManager.sleep_grammar | ||
| ExclusiveManager.sleeping = state | ||
| grammar.set_exclusive(state) | ||
| get_engine().process_grammars_context() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| from dragonfly import Dictation, MappingRule, Choice, Function | ||
| from castervoice.lib.actions import Text | ||
|
|
||
| from castervoice.lib.ctrl.mgr.rule_details import RuleDetails | ||
| from castervoice.lib.merge.state.short import R | ||
|
|
||
| from castervoice.lib.ctrl.mgr.engine_manager import EngineModesManager | ||
|
|
||
| class CasterMicRule(MappingRule): | ||
| mapping = { | ||
| "caster <mic_mode>": Function(lambda mic_mode: EngineModesManager.set_mic_mode(mode=mic_mode)), | ||
| } | ||
| extras = [ | ||
| Choice( | ||
| "mic_mode", { | ||
| "off": "off", | ||
| "on": "on", | ||
| "sleep": "sleeping", | ||
| }), | ||
| ] | ||
| defaults = {} | ||
|
|
||
| def get_rule(): | ||
| return CasterMicRule, RuleDetails(name="caster mic mode", transformer_exclusion=True) | ||
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get this when saying "caster off"
Caster: 'off' is not valid. set_mic_state modes are: 'off' - DPI Only, 'on', 'sleeping'Is that deliberate? We should also have "caster wake" as an option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@LexiconCode thanks for doing this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes it is intentional ('off' - DPI Only). For other engines besides DPI/DNS to have an off state they need a method to turn the microphone on without voice activation like a keyboard shortcut/GUI toggle. DPI/DNS has that capability. Why
sleepwithsleeping?There's a difference between
sleepthe command andsleepingwhich is passed as a parameter.sleepingis what Natlink uses as a string to put DNS to sleepnatlink.setMicState(mode).The mode needs to be checked in EngineModesManager because they can be set in the caster settings for the engine to start up in a certain mode. Because the user is defining a string (a mode) within the settings which could be anything must be checked against valid modes.
"caster wake" would be possible but I don't really know why unless there is recognition issues. It's nice to provide something that's concise and intuitive. "wake" does feel natural but that's because of my experience of DNS. I would think most people without that experience would think "on" "off" "sleep".
One other bit of feedback is that the dictation element needs to be weighted for kaldi in
sleep_rule?