Dream Art combines inceptionism/DeepDream with "[A Neural Algorithm of Artistic Style][neural-art-arxiv]." The following examples were created with create-outputs.sh and show an input content image followed by the DeepDream content image, and then pairs of input artistic styles and the DeepDream content image with the style applied. See the gallery for more examples.
TODO
Tweet the style and source image to
@brandondamos
with the hashtag #DreamArtRequest and I'll manually
generate images for you.
If there's enough interest, I'll create a bot to
automatically process images with #DreamArtRequest
on my server.
Inceptionism is introduced in Google Research's blog post and visualizes an artificial neural network by enhancing input images to elicit an interpretation. The following example from the blog post. See the original gallery or #DeepDream for more examples.
The current implementations are google/deepdream which uses Caffe and eladhoffer/DeepDream.torch which uses Torch.
'[A Neural Algorthm of Artistic Style][neural-art-arxiv]' is a paper by Leon Gatys, Alexander Ecker, and Matthias Bethge released as a preprint on August 26, 2015. The current implementations are [jcjohnson/neural-style][neural-style] and [kaishengtai/neuralart][neural-art], which both use Torch.
Nontrivial toolchains and system configuration make the barrier to running current implementations high. This repository lowers the barrier by combining DeepDream.torch with [neural-style][neural-style] into Torch applications application that share the same model and run on the CPU and GPU.
Ideally, both of these implementations should ship on luarocks and provide a command-line interface and library that nicely load the models. When they do, this repository will be obsolete, but until then, this repository glues them together.
The following is from [jcjohnson/neural-style][neural-style].
Dependencies:
Optional dependencies:
- CUDA 6.5+
- cudnn.torch
NOTE: If your machine does not have CUDA installed, then you may need to install loadcaffe manually like this:
git clone https://github.com/szagoruyko/loadcaffe.git
# Edit the file loadcaffe/loadcaffe-1.0-0.rockspec
# Delete lines 21 and 22 that mention cunn and inn
luarocks install loadcaffe/loadcaffe-1.0-0.rockspec
After installing dependencies, you'll need to run the following script to download the VGG model:
sh models/download_models.sh
This will download the original VGG-19 model. Leon Gatys has graciously provided the modified version of the VGG-19 model that was used in their paper; this will also be downloaded. By default the original VGG-19 model is used.
See th deepdream.lua -help and th neural-style.lua -help for the most updated docs.
dream-art(master*)$ ./deepdream.lua -help
-content_image Content target image [examples/inputs/tubingen.jpg]
-gpu Zero-indexed ID of the GPU to use; for CPU mode set -gpu = -1 [0]
-num_iter [100]
-num_octave [8]
-octave_scale [1.4]
-end_layer [32]
-clip [true]
-proto_file [models/VGG_ILSVRC_19_layers_deploy.prototxt]
-model_file [models/VGG_ILSVRC_19_layers.caffemodel]
-backend nn|cudnn [nn]
-output_image [out.png]
dream-art(master*)$ ./neural-style.lua -help
-style_image Style target image [examples/inputs/seated-nude.jpg]
-content_image Content target image [examples/inputs/tubingen.jpg]
-image_size Maximum height / width of generated image [512]
-gpu Zero-indexed ID of the GPU to use; for CPU mode set -gpu = -1 [0]
-content_weight [5]
-style_weight [100]
-tv_weight [0.001]
-num_iterations [1000]
-init random|image [random]
-print_iter [50]
-save_iter [100]
-output_image [out.png]
-style_scale [1]
-pooling max|avg [max]
-proto_file [models/VGG_ILSVRC_19_layers_deploy.prototxt]
-model_file [models/VGG_ILSVRC_19_layers.caffemodel]
-backend nn|cudnn [nn]
- How long does it take to process a single image? About 20 minutes with a Tesla K40.
- How much memory does this use? This depends on the neural network model and image size. On average, 4GB.
All portions are MIT licensed by Brandon Amos unless otherwise noted.
This project uses and modifies the following open source projects and resources. Modifications remain under the original license.
| Project | Modified | License |
|---|---|---|
| DeepDream.torch | Yes | MIT |
| [jcjohnson/neural-style][neural-style] | Yes | MIT |
[neural-art-arxiv]: http://arxiv.org/abs/1508.06576) [neural-style]: https://github.com/jcjohnson/neural-style [neural-art]: https://github.com/kaishengtai/neuralart