waifu2x

Image Super-Resolution for Anime-Style Art

Subscribe to updates I use waifu2x


Statistics on waifu2x

Number of watchers on Github 10626
Number of open issues 60
Average time to close an issue 29 days
Main language Lua
Average time to merge a PR 2 days
Open pull requests 4+
Closed pull requests 5+
Last commit 7 months ago
Repo Created over 3 years ago
Repo Last Updated 6 months ago
Size 512 MB
Organization / Authornagadomi
Latest Releasev0.13.1
Contributors6
Page Updated
Do you use waifu2x? Leave a review!
View open issues (60)
View waifu2x activity
View on github
Latest Open Source Launches
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating waifu2x for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

waifu2x

Image Super-Resolution for Anime-style art using Deep Convolutional Neural Networks. And it supports photo.

The demo application can be found at http://waifu2x.udp.jp/ .

Summary

Click to see the slide show.

slide

References

waifu2x is inspired by SRCNN [1]. 2D character picture (HatsuneMiku) is licensed under CC BY-NC by piapro [2].

  • [1] Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang, Image Super-Resolution Using Deep Convolutional Networks, http://arxiv.org/abs/1501.00092
  • [2] For Creators, http://piapro.net/en_for_creators.html

Public AMI

Region: us-east-1 (N.Virginia)
AMI ID: ami-b9be23ae
AMI NAME: waifu2x-server 20160807
Instance Type: g2.2xlarge
OS: Ubuntu 14.04
User: ubuntu
Created at: 2016-08-07

See ~/README.md

Please update the git repo first.

git pull

Third Party Software

Third-Party

If you are a windows user, I recommend you to use waifu2x-caffe(Just download from releases tab) or waifu2x-conver-cpp.

Dependencies

Hardware

  • NVIDIA GPU

Platform

LuaRocks packages (excludes torch7's default packages)

  • lua-csnappy
  • md5
  • uuid
  • csvigo
  • turbo

Installation

Setting Up the Command Line Tool Environment

(on Ubuntu 14.04)

Install CUDA

See: NVIDIA CUDA Getting Started Guide for Linux

Download CUDA

sudo dpkg -i cuda-repo-ubuntu1404_7.5-18_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Install Package

sudo apt-get install libsnappy-dev
sudo apt-get install libgraphicsmagick1-dev

Install Torch7

See: Getting started with Torch, and #222

And install luarocks packages.

luarocks install graphicsmagick # upgrade
luarocks install lua-csnappy
luarocks install md5
luarocks install uuid
luarocks install csvigo

# if you need to use web application
PREFIX=$HOME/torch/install luarocks install turbo

# if you need to use cuDNN library. cuDNN is required.
luarocks install cudnn

Getting waifu2x

git clone --depth 1 https://github.com/nagadomi/waifu2x.git

Validation

Testing the waifu2x command line tool.

th waifu2x.lua

Web Application

th web.lua

View at: http://localhost:8812/

Command line tools

Notes: If you have cuDNN library, than you can use cuDNN with -force_cudnn 1 option. cuDNN is too much faster than default kernel.

Noise Reduction

th waifu2x.lua -m noise -noise_level 1 -i input_image.png -o output_image.png
th waifu2x.lua -m noise -noise_level 0 -i input_image.png -o output_image.png
th waifu2x.lua -m noise -noise_level 2 -i input_image.png -o output_image.png
th waifu2x.lua -m noise -noise_level 3 -i input_image.png -o output_image.png

2x Upscaling

th waifu2x.lua -m scale -i input_image.png -o output_image.png

Noise Reduction + 2x Upscaling

th waifu2x.lua -m noise_scale -noise_level 1 -i input_image.png -o output_image.png
th waifu2x.lua -m noise_scale -noise_level 0 -i input_image.png -o output_image.png
th waifu2x.lua -m noise_scale -noise_level 2 -i input_image.png -o output_image.png
th waifu2x.lua -m noise_scale -noise_level 3 -i input_image.png -o output_image.png

Batch conversion

find /path/to/imagedir -name "*.png" -o -name "*.jpg" > image_list.txt
th waifu2x.lua -m scale -l ./image_list.txt -o /path/to/outputdir/prefix_%d.png

The output format supports %s and %d(e.g. %06d). %s will be replaced the basename of the source filename. %d will be replaced a sequence number. For example, when input filename is piyo.png, %s_%03d.png will be replaced piyo_001.png.

See also th waifu2x.lua -h.

Using photo model

Please add -model_dir models/photo to command line option, if you want to use photo model. For example,

th waifu2x.lua -model_dir models/photo -m scale -i input_image.png -o output_image.png

Video Encoding

* avconv is alias of ffmpeg on Ubuntu 14.04.

Extracting images and audio from a video. (range: 00:09:00 ~ 00:12:00)

mkdir frames
avconv -i data/raw.avi -ss 00:09:00 -t 00:03:00 -r 24 -f image2 frames/%06d.png
avconv -i data/raw.avi -ss 00:09:00 -t 00:03:00 audio.mp3

Generating a image list.

find ./frames -name "*.png" |sort > data/frame.txt

waifu2x (for example, noise reduction)

mkdir new_frames
th waifu2x.lua -m noise -noise_level 1 -resume 1 -l data/frame.txt -o new_frames/%d.png

Generating a video from waifu2xed images and audio.

avconv -f image2 -framerate 24 -i new_frames/%d.png -i audio.mp3 -r 24 -vcodec libx264 -crf 16 video.mp4

Train Your Own Model

Note1: If you have cuDNN library, you can use cudnn kernel with -backend cudnn option. And, you can convert trained cudnn model to cunn model with tools/rebuild.lua.

Note2: The command that was used to train for waifu2x's pretraind models is available at appendix/train_upconv_7_art.sh, appendix/train_upconv_7_photo.sh. Maybe it is helpful.

Data Preparation

Genrating a file list.

find /path/to/image/dir -name "*.png" > data/image_list.txt

You should use noise free images. In my case, waifu2x is trained with 6000 high-resolution-noise-free-PNG images.

Converting training data.

th convert_data.lua

Train a Noise Reduction(level1) model

mkdir models/my_model
th train.lua -model_dir models/my_model -method noise -noise_level 1 -test images/miku_noisy.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise -noise_level 1 -i images/miku_noisy.png -o output.png

You can check the performance of model with models/my_model/noise1_best.png.

Train a Noise Reduction(level2) model

th train.lua -model_dir models/my_model -method noise -noise_level 2 -test images/miku_noisy.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise -noise_level 2 -i images/miku_noisy.png -o output.png

You can check the performance of model with models/my_model/noise2_best.png.

Train a 2x UpScaling model

th train.lua -model upconv_7 -model_dir models/my_model -method scale -scale 2 -test images/miku_small.png
# usage
th waifu2x.lua -model_dir models/my_model -m scale -scale 2 -i images/miku_small.png -o output.png

You can check the performance of model with models/my_model/scale2.0x_best.png.

Train a 2x and noise reduction fusion model

th train.lua -model upconv_7 -model_dir models/my_model -method noise_scale -scale 2 -noise_level 1 -test images/miku_small.png
# usage
th waifu2x.lua -model_dir models/my_model -m noise_scale -scale 2 -noise_level 1 -i images/miku_small.png -o output.png

You can check the performance of model with models/my_model/noise1_scale2.0x_best.png.

Docker

Requires nvidia-docker.

docker build -t waifu2x .
nvidia-docker run -p 8812:8812 waifu2x th web.lua
nvidia-docker run -v `pwd`/images:/images waifu2x th waifu2x.lua -force_cudnn 1 -m scale -scale 2 -i /images/miku_small.png -o /images/output.png

Note that running waifu2x in without JIT caching is very slow, which is what would happen if you use docker. For a workaround, you can mount a host volume to the CUDA_CACHE_PATH, for instance,

nvidia-docker run -v $PWD/ComputeCache:/root/.nv/ComputeCache waifu2x th waifu2x.lua --help
waifu2x open issues Ask a question     (View All Issues)
  • almost 2 years [suggest] Residual Learning?
  • about 2 years non-deterministic result when using cuDNN and upconv_7
  • about 2 years waifu displays in the wrong language
  • about 2 years Mesh style artifacts on new scale 2x model
  • over 2 years [Idea] Training using vector graphics
  • over 2 years [Question] Is it possible to freely scale? example: 2.5x, 3.0x, 4.1x etc.
  • over 2 years Waifu2x in real time
  • over 2 years Method Not Allowed
  • over 2 years Two questions about training process
  • over 2 years high performance version?
  • over 2 years Add TTA mode in waifu2x.udp.jp?
  • over 2 years Gif error
  • over 2 years architecture question
  • over 2 years [Overview] Models differences
  • over 2 years Use original filename for output file.
  • over 2 years Failure to work on my HP z400 & HP z820
  • over 2 years photo model saturation
  • almost 3 years Failing on Mac OS X 10.11
  • almost 3 years Feature suggestion: Dither removal
  • almost 3 years Large file in git repo
  • over 3 years If you plan to share anime episodes after upscaling using this, please read.
  • over 3 years Perceptually uniform colorspace processing?
  • over 3 years web.lua crashed after 10 hours runtime
  • over 3 years Training questions
waifu2x open pull requests (View All Pulls)
  • Update ru.yml
  • Added German Translation
  • BLEH
  • Update fr.yml
waifu2x list of languages used
waifu2x latest release notes
v0.13.1 v0.13.1
  • Fixed mesh-like artifact on art model #125
  • Fixed wrong format in upconv_7/art/noise0_scale2.0x.t7 (cudnn -> cunn)
v0.13 v0.13
  • Added new upscaling model (models/upconv_7)
  • Added support for user method (universal filter)
Other projects in Lua