JWQL Logging Function: The Why's and The How

What is logging_functions.py and why should I use it?

logging_functions.py is a module that will allow us to automatically log the execution of any script it is implemented in.

This is useful for tracking the use of and successfulness of our modules. It can also be useful in updating and trouble-shooting scripts as well while we are developing them.

The following information will be tracked in the log file:

• Environment information from the system that ran the script.
• A timestamped logging of anything produced by the script.
• Error tracebacks in the event the script crashes.
• Elapsed CPU and Real Time tracking of the scripts.
• Track how long a query takes to run.

How do I use it?

At the top of your script you will need to import the various logging functions like this:

import logging
from jwql.logging.logging_functions import configure_logging
from jwql.logging.logging_functions import log_info
from jwql.logging.logging_functions import log_fail

Then, above the `main()' function of your script you need to add the log fail and log info decorators like such:

@log_fail
@log_info 
def this_is_my_module_main():
    ...

Lastly, you need to place some config and logging calls in the end portion as well:

if __name__ == '__main__':

    module = os.path.basename(__file__).strip('.py')
    configure_logging(module)

    this_is_my_module_main()

Quick warning! Beware 'print' statements!

You will also need to double check that you change any print() statements from your function into logging.info() statements. This allows the logging_functions to run fully AND keeps track of that information in your log file, i.e. it will print that information to your log file.

Example:

Instead of this:

print('Starting the function run: ')
...
print(file_list_the_function_made)
...
print('Finished with the list')

Have this information added in this way:

logging.info('Starting the function run: ')
...
logging.info(file_list_the_function_made)
...
logging.info('Finished with the list')

Where will I find my log files?

The log files will be available in the jwql central storage area under /logs/<|dev|test|ops>/<module>/<module_log_filename>.

Why isn't it working? Some basic trouble-shooting:

1. Does your module have a folder in /logs/dev|test|ops/? Are the permissions for that folder open?

2. Did you place the decorators in the proper location? They need to be attached to your main function that runs your script by being place just above that.

3. Did you clean up all the print statements to instead be logging statements?

4. Is there a suggestion for this trouble-shooting that should be included here? A problem or mistake you ran into that other people may have as well?