Internal display support

* Issues with power management

- Was unable to change off time due to copy'n'paste
- Was unable to turn off scheduling

* Initial version of RGB888<->RGB565 conversion tool

* Initial change to support non-HDMI displays

* Reframing limits

Reframing should not happen if zoomed area is smaller than 20px on each
side of the frame. Looks silly.

* Major rewrite to tvservice handling

No longer deals with pixels, uses tv service string to deduce all
values, avoids trying to use non-existant resolution if TV is unplugged.

* Handle lack of ANY display

photoframe should not crash if a display is missing, instead UX should
show the issue at hand.

* Stop exception when no colorsensor is available

* Handle broken json from google

Sometimes downloads fail and the JSON is broken, which we now deal
with, but the rest of the code doesn't. This fixes the issue.

* Initial support for uploading drivers

Also enables listing and activating them

* Added support for uploading drivers

* Better handling of no display at all

* Reboot/power off now works as expcted

Turns out that replacing the HTML during async call will cancel
out that call. Now we just replace the body instead of the document.

* Add driver documentation

* Removed usage of old setting name

resolution is gone, replaced with tvservice

* Relocate all config files into separate folder

* Corrected README.md since it used old paths

Also clarified the need for chmod

README.md

@@ -223,7 +223,9 @@ GND -> Pin 9 (GND)
 You also need to tell your RPi3 to enable the I2C bus, start the `raspi-config` and go to submenu 5 (interfaces) and select I2C and enable it.
 
 Once all this is done, you have one more thing left to do before rebooting, you need to download the imagemagick script that will adjust the image,
-please visit http://www.fmwconcepts.com/imagemagick/colortemp/index.php and download and store it as `colortemp.sh` inside `/root/`.
+please visit http://www.fmwconcepts.com/imagemagick/colortemp/index.php and download and store it as `colortemp.sh` inside `/root/photoframe_config`.
+
+Don't forget to make it executable by `chmod +x /root/photoframe_config/colortemp.sh` or it will still not work.
 
 You're done! Reboot your RPi3 (So I2C gets enabled) and from now on, all images will get adjusted to match the ambient color temperature.
 

display-drivers/README.md

@@ -0,0 +1,93 @@
+# Display Drivers
+
+Photoframe now supports uploading and enabling of internal displays for the Raspberry Pi family,
+the only requirement is that it can be supported by the currently used kernel and modules.
+
+Since a lot of the smaller displays rely on the built-in fbtft driver, it means that in many
+cases, all you really need is a DeviceTree Overlay, essentially configuration files for the
+device driver so it knows how to talk to the new display.
+
+## What's included
+
+Today, only the waveshare 3.5" IPS (model B) is provided since that was my development system.
+But you can create and share these display "drivers" easily yourself.
+
+## How to write a display driver package
+
+Start with an empty folder, copy the necessary files for it to work, usually one or two files
+ending in `.dtb` or `.dtbo`.
+
+Next, create a file called `INSTALL` (yes, all caps, important) in the same folder. Open the
+file and create the following structure:
+
+```
+[install]
+
+[options]
+```
+
+### The install section
+This is a very simple `key/value` pair setup. First part (key) refers to the file included in the
+package. The path to the file is based on the location of the `INSTALL` file. You can use 
+sub-directories if you need to, but if you do so, they must adhere to the same rule.
+
+The value part refers to where the file should be copied when activated. Typically this is
+somewhere in `/boot/`.
+
+For example, in the waveshare case, this section looks like this:
+```
+[install]
+waveshare35b-overlay.dtb=/boot/overlays/waveshare35b.dtbo
+waveshare35b-overlay.dtb=/boot/overlays/waveshare35b-overlay.dtb
+```
+As you can see, the key here is used multiple times, this is because they place this file in two
+locations with different names (but it's the same file).
+
+NOTE! The installer will NOT create any directories when activating.
+
+### The options section
+
+This is also a `key/value` setup, but unlike the `install` section, here the key is UNIQUE. If you
+define a key multiple times, only the last definition will be used.
+
+At the very least, this section holds the key `dtoverlay` which is the `/boot/config.txt` keyword
+for pointing out an overlay to use. But you can add as many things as you'd like (some DPI displays
+require a multitude of key/value pairs).
+
+In the waveshare 3.5" display case, all it does is point out the overlay:
+```
+[options]
+dtoverlay=waveshare35b
+```
+
+## Saving the display driver package
+
+Once you have written your `INSTALL` file and added the needed files to the folder you
+created earlier, all you need to do now is create a zip file out of the contents and give
+the file a nice name (like, `waveshare35b.zip`) since that's the name used to identify
+the driver.
+
+## I updated my driver, now what?
+
+Simply upload it again. The old driver will be deleted and replaced with the new one.
+
+## This all seem complicated, do you have an example?
+
+Sure, just unzip the `waveshare35b.zip` and look at it for guidance.
+
+## What is `manifest.json` ?
+
+That's a generated file by photoframe which it creates upon installing a driver. You can
+create sub-directories in `display-drivers` with pre-processed drivers which will then be
+available by default when installing photoframe.
+
+Note that if you install a driver with the same name as one of the provided ones, the new
+driver will take priority
+
+## Known gotchas
+
+If you install a driver which you're already using, you need to switch to HDMI and back to
+force update the active driver (and no, no need to reboot when going to HDMI, only when
+you go back to your updated driver). 
+
+This will eventually be fixed.

frame.py

@@ -38,14 +38,26 @@
 from modules.oauth import OAuth
 from modules.slideshow import slideshow
 from modules.colormatch import colormatch
+from modules.drivers import drivers
 
 void = open(os.devnull, 'wb')
+# Supercritical, since we store all photoframe files in a subdirectory, make sure to create it
+if not os.path.exists('/root/photoframe_config'):
+	try:
+		os.mkdir('/root/photoframe_config')
+	except:
+		logging.exception('Unable to create configuration directory, cannot start')
+		sys.exit(255)
+elif not os.path.isdir('/root/photoframe_config'):
+	logging.error('/root/photoframe_config isn\'t a folder, cannot start')
+	sys.exit(255)
 
 import requests
 from requests_oauthlib import OAuth2Session
-from flask import Flask, request, redirect, session, url_for, abort
+from flask import Flask, request, redirect, session, url_for, abort, flash
 from flask.json import jsonify
 from flask_httpauth import HTTPBasicAuth
+from werkzeug.utils import secure_filename
 
 # used if we don't find authentication json
 class NoAuth:
@@ -74,8 +86,9 @@ def wrap(*args, **kwargs):
 logging.getLogger('urllib3').setLevel(logging.ERROR)
 
 app = Flask(__name__, static_url_path='')
+app.config['UPLOAD_FOLDER'] = '/tmp/'
 user = None
-userfiles = ['/boot/http-auth.json', '/root/http-auth.json']
+userfiles = ['/boot/http-auth.json', '/root/photoframe_config/http-auth.json']
 
 for userfile in userfiles:
 	if os.path.exists(userfile):
@@ -126,34 +139,31 @@ def cfg_keyvalue(key, value):
 			return
 
 	if request.method == 'PUT':
+		status = True
 		if key == "keywords":
 			# Keywords has its own API
 			abort(404)
 			return
 		settings.setUser(key, value)
 		settings.save()
-		if key in ['width', 'height', 'depth', 'tvservice']:
-			display.setConfiguration(settings.getUser('width'), settings.getUser('height'), settings.getUser('depth'), settings.getUser('tvservice'))
-			display.enable(True, True)
+		if key in ['display-driver']:
+			drv = settings.getUser('display-driver')
+			if drv == 'none':
+				drv = None
+			if not drivers.activate(drv):
+				settings.setUser('display-driver', 'none')
+				status = False
 		if key in ['timezone']:
 			# Make sure we convert + to /
 			settings.setUser('timezone', value.replace('+', '/'))
 			helper.timezoneSet(settings.getUser('timezone'))
-		if key in ['resolution']:
-			# This one needs some massaging, we essentially deduce all settings from a string (DMT/CEA CODE HDMI)
-			items = settings.getUser('resolution').split(' ')
-			logging.debug('Items: %s', repr(items))
-			resolutions = display.available()
-			for res in resolutions:
-				if res['code'] == int(items[1]) and res['mode'] == items[0]:
-					logging.debug('Found this item: %s', repr(res))
-					settings.setUser('width',  res['width'])
-					settings.setUser('height', res['height'])
-					settings.setUser('depth', 32)
-					settings.setUser('tvservice', value)
-					display.setConfiguration(settings.getUser('width'), settings.getUser('height'), settings.getUser('depth'), settings.getUser('tvservice'))
-					display.enable(True, True)
-					break
+		if key in ['resolution', 'tvservice']:
+			width, height, tvservice = display.setConfiguration(value)
+			settings.setUser('tvservice', tvservice)
+			settings.setUser('width',  width)
+			settings.setUser('height', height)
+			settings.save()
+			display.enable(True, True)
 		if key in ['display-on', 'display-off']:
 			timekeeper.setConfiguration(settings.getUser('display-on'), settings.getUser('display-off'))
 		if key in ['autooff-lux', 'autooff-time']:
@@ -163,7 +173,7 @@ def cfg_keyvalue(key, value):
 		if key in ['shutdown-pin']:
 			powermanagement.stopmonitor()
 			powermanagement = shutdown(settings.getUser('shutdown-pin'))
-		return jsonify({'status':True})
+		return jsonify({'status':status})
 
 	elif request.method == 'GET':
 		if key is None:
@@ -212,7 +222,7 @@ def cfg_oauth_info():
 		abort(500)
 	data = request.json['web']
 	oauth.setOAuth(data)
-	with open('/root/oauth.json', 'wb') as f:
+	with open('/root/photoframe_config/oauth.json', 'wb') as f:
 		json.dump(data, f);
 	return jsonify({'result' : True})
 
@@ -249,6 +259,9 @@ def cfg_details(about):
 		response = app.make_response(image)
 		response.headers.set('Content-Type', mime)
 		return response
+	elif about == 'drivers':
+		result = drivers.list().keys()
+		return jsonify(result)
 	elif about == 'timezone':
 		result = helper.timezoneList()
 		return jsonify(result)
@@ -263,6 +276,28 @@ def cfg_details(about):
 
 	abort(404)
 
+@app.route('/custom-driver', methods=['POST'])
+@auth.login_required
+def upload_driver():
+	if request.method == 'POST':
+		# check if the post request has the file part
+		if 'driver' not in request.files:
+			logging.error('No file part')
+			abort(405)
+		file = request.files['driver']
+		# if user does not select file, browser also
+		# submit an empty part without filename
+		if file.filename == '' or not file.filename.lower().endswith('.zip'):
+			logging.error('No filename or invalid filename')
+			abort(405)
+		filename = os.path.join('/tmp/', secure_filename(file.filename))
+		file.save(filename)
+		if drivers.install(filename):
+			return ''
+		else:
+			abort(500)
+	abort(405)
+
 @app.route("/link")
 @auth.login_required
 def oauth_step1():
@@ -295,20 +330,27 @@ def web_template(file):
 	return app.send_static_file('template/' + file)
 
 settings = settings()
+drivers = drivers()
+display = display()
+
 if not settings.load():
 	# First run, grab display settings from current mode
 	current = display.current()
-	logging.info('No display settings, using: %s' % repr(current))
-	settings.setUser('tvservice', '%s %s HDMI' % (current['mode'], current['code']))
-	settings.setUser('width', int(current['width']))
-	settings.setUser('height', int(current['height']))
-	settings.save()
-
+	if current is not None:
+		logging.info('No display settings, using: %s' % repr(current))
+		settings.setUser('tvservice', '%s %s HDMI' % (current['mode'], current['code']))
+		settings.save()
+	else:
+		logging.info('No display attached?')
 if settings.getUser('timezone') == '':
 	settings.setUser('timezone', helper.timezoneCurrent())
 	settings.save()
 
-display = display(settings.getUser('width'), settings.getUser('height'), settings.getUser('depth'), settings.getUser('tvservice'))
+width, height, tvservice = display.setConfiguration(settings.getUser('tvservice'))
+settings.setUser('tvservice', tvservice)
+settings.setUser('width',  width)
+settings.setUser('height', height)
+settings.save()
 
 # Force display to desired user setting
 display.enable(True, True)
@@ -333,8 +375,8 @@ def oauthSetToken(token):
 
 oauth = OAuth(settings.get('local-ip'), oauthSetToken, oauthGetToken)
 
-if os.path.exists('/root/oauth.json'):
-	with open('/root/oauth.json') as f:
+if os.path.exists('/root/photoframe_config/oauth.json'):
+	with open('/root/photoframe_config/oauth.json') as f:
 		data = json.load(f)
 	if 'web' in data: # if someone added it via command-line
 		data = data['web']

modules/colormatch.py

@@ -133,7 +133,11 @@ def run(self):
 		# I2C address 0x29
 		# Register 0x12 has device ver.
 		# Register addresses must be OR'ed with 0x80
-		bus.write_byte(0x29,0x80|0x12)
+		try:
+			bus.write_byte(0x29,0x80|0x12)
+		except:
+			logging.info('ColorSensor not available')
+			return
 		ver = bus.read_byte(0x29)
 		# version # should be 0x44
 		if ver == 0x44: