Implement scoped parameters

[robparrott]

We want a user to be able to set a parameter on a provider, a cloudlet or specific blueprint, as well as the current options which are "global" and "this session".

Suggest command line looks like:

nepho parameter [--provider aws] set | get | unset key [value]
nepho parameter [--cloudlet my-cloudlet ]  set | get | unset key [value]
nepho parameter [--cloudlet my-cloudlet ]  [--blueprint my-blueprint] set | get | unset key [value]

I suggest implementing by structuring the parameters dictionary as

parameters:
  common:
       key1 : value1
       key2 : value2
  provider:
       aws:
         key1 : value1
         key2 : value2 
       vagrant:
         key : value
  cloudlet:
    cloudlet1:
      common:
        key1 : value1
        key2 : value2  
      blueprint1:
        key1 : value1
        key2 : value2 
    cloudlet2:
       .....

Given that YAML <----> dict() is robust, coding this up should be straightforward. We'll want to be sure to cleanup unneeded dicts()

[hakamadare]

i want this done badly enough to start working on it :)

[agperson]

Let's talk at some point about the config vs param divide and whether the two sets of data should be handled the same way. I'm slightly worried that moving our config management out of the Cement library could break things in the future. We should either subclass our config handler from the Cement IConfig handler or make a clean break.

[hakamadare]

so, i think that a must-have feature of parameters is that they be organized in a hierarchy (i.e. this feature). given that iirc IConfig wants to write flat ini-files, i think that's an argument for putting config in IConfig and implementing our own parameter handling with a YAML backend.

[robparrott]

That’s kinda the point of the current config implementation.

[agperson]

I think keeping CLI/app config separate from scoped parameters makes sense. It should't be a very big or disruptive change at this point, and it keeps the params hierarchy cleaner. The only downside I see is if it becomes confusing to decide what is a config vs a parameter.

But I don't really think that modifying the config via the app is necessary, just modify the config file. Params, on the other hand, do make sense to be modifiable by the app.

[robparrott]

Actually, I don’t agree with you about modifying the configs via the app. I think it’s a very useful capability to allow this, and one which the IAM folks were interested in. In particular, for setting up there own registry URLs and repos that aren’t on github.

Separating them into two separate files is fine, but I don’t see the driving reason for it.

On November 15, 2013 at 3:33:30 PM, Daniel Silverman ([email protected]) wrote:

I think keeping CLI/app config separate from scoped parameters makes sense. It should't be a very big or disruptive change at this point, and it keeps the params hierarchy cleaner. The only downside I see is if it becomes confusing to decide what is a config vs a parameter.

But I don't really think that modifying the config via the app is necessary, just modify the config file. Params, on the other hand, do make sense to be modifiable by the app.

—
Reply to this email directly or view it on GitHub.

[hakamadare]

i'm with @robparrott on this one; for the Stewies of the world, modifying a text file can be scary and fraught with peril (fun with line endings, anyone?), and so i think it behooves us to give them a way to do all the configuration they need via the app.

i think the driving force is the idea that Cement's native config manager is way more robust than anything that we're going to implement in any short period of time, and so we should leverage it as much as possible. our only configuration-related use case where it's not a good fit is the parameters, since we need them to be hierarchical; thus, we should use the native config management as much as possible, and then do something with YAML for the parameters.

[agperson]

Ok, so we will:

• Move configs back into the IConfig class. Configs are Nepho-specific (paths to local directories, output formats, registry URLs) and not passed into contexts. Parameters are anything that is passed into a context.
• Parameters will be stored in YAML, in a location specified in the config.
• Parameters will be scoped as:
  Global -> Provider -> Cloudlet -> Blueprint
• The parameter class will handle scoping and return the one proper value, or None
• Both cloudlet.yaml and blueprint.yaml will have an optional parameter list with optional defaults
• If a default is not specified, and the parameter is not present, the user will be prompted and the value saved at blueprint scope (or, for now, receive an informative error).
• If a parameter is not specified in the union of cloudlet.yaml and blueprint.yaml, it will not be included in the context.
• If providers need params (ex: aws_access_key_id) they will get them from provider scope or global scope only, and will provide their own errors or prompts.