char-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow

Star full 4f7b624809470f25b6493d5a7b30d9b9cb905931146e785d67c86ef0c205a402Star full 4f7b624809470f25b6493d5a7b30d9b9cb905931146e785d67c86ef0c205a402Star full 4f7b624809470f25b6493d5a7b30d9b9cb905931146e785d67c86ef0c205a402Star full 4f7b624809470f25b6493d5a7b30d9b9cb905931146e785d67c86ef0c205a402Star half bd79095782ee4930099175e5ce7f4c89fa3ddabcd56fffcc7c74f6f2a2d46b27 (3 ratings)
Rated 4.5 out of 5
Subscribe to updates I use char-rnn-tensorflow


Statistics on char-rnn-tensorflow

Number of watchers on Github 1850
Number of open issues 45
Average time to close an issue 17 days
Main language Python
Average time to merge a PR 13 days
Open pull requests 29+
Closed pull requests 10+
Last commit 10 months ago
Repo Created over 2 years ago
Repo Last Updated 4 months ago
Size 491 KB
Organization / Authorsherjilozair
Contributors2
Page Updated
Do you use char-rnn-tensorflow? Leave a review!
View open issues (45)
View char-rnn-tensorflow activity
View on github
Latest Open Source Launches
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating char-rnn-tensorflow for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

char-rnn-tensorflow

Join the chat at https://gitter.im/char-rnn-tensorflow/Lobby Coverage Status Build Status

Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow.

Inspired from Andrej Karpathy's char-rnn.

Requirements

Basic Usage

To train with default parameters on the tinyshakespeare corpus, run python train.py. To access all the parameters use python train.py --help.

To sample from a checkpointed model, python sample.py.

Datasets

You can use any plain text file as input. For example you could download The complete Sherlock Holmes as such:

cd data
mkdir sherlock
cd sherlock
wget https://sherlock-holm.es/stories/plain-text/cnus.txt
mv cnus.txt input.txt

Then start train from the top level directory using python train.py --data_dir=./data/sherlock/

A quick tip to concatenate many small disparate .txt files into one large training file: ls *.txt | xargs -L 1 cat >> input.txt

Tensorboard

To visualize training progress, model graphs, and internal state histograms: fire up Tensorboard and point it at your log_dir. E.g.:

$ tensorboard --logdir=./logs/

Then open a browser to http://localhost:6006 or the correct IP/Port specified.

Roadmap

  • [ ] Add explanatory comments
  • [ ] Expose more command-line arguments
  • [ ] Compare accuracy and performance with char-rnn
  • [ ] More Tensorboard instrumentation

Contributing

Please feel free to:

  • Leave feedback in the issues
  • Open a Pull Request
  • Join the gittr chat
  • Share your success stories and data sets!
char-rnn-tensorflow open issues Ask a question     (View All Issues)
  • over 1 year Interface to save checkpoint?
  • over 1 year How to reduce GPU memory?
  • over 1 year Enhancement - Dockerfile + GIST
  • over 1 year Tensorflow 0.11 Tuple issue
  • over 1 year loop function
  • almost 2 years W tensorflow/core/framework/op_kernel.cc:909] Resource exhausted:OOM when allocating tensor with shape[1250,25670]
  • about 2 years Char sequence probability
  • about 2 years Support Unicode Input Files
  • about 2 years No dropout option?
  • about 2 years No validation/test?
  • about 2 years Compat issues with latest TF
  • about 2 years MemoryError
  • about 2 years Why we need ydata[-1] = xdata[0]?
  • about 2 years Tuning the temperature
  • over 2 years weighted_pick() can return invalid index
  • over 2 years create_batches in TextLoader in utils.py doesn't seem to transform the data into batches correctly
  • over 2 years Can not convert a list into a Tensor or Operation
  • over 2 years create_batches() from utils.py throws error
  • over 2 years IndexError is out of Bounce
  • over 2 years How to calculate prob of a new sentence.
  • over 2 years Curious...
  • over 2 years Not a big deal but your output files are not part of .gitignore
  • over 2 years Sampling probablilities do not sum to 1
char-rnn-tensorflow open pull requests (View All Pulls)
  • Changes which might be interesting
  • when training, model can be initialized from previously saved model
  • Better error message
  • Added the last save
  • Implement temperature
  • Dropouts
  • Open pickled files in binary mode
  • Allow for infinite sampling streams.
  • Not to create unnessary tensorflow variables when sampling
  • More portable way of executing tensor
  • Fix #47
  • add a script to compute the perplexity of test data
  • BRNN + perplexity evaluation
  • Adding a bidirectional neural network
  • Replace deprecated Tensorflow functions
  • Make save_dir if doesn't exist
  • Simple tests on Travis CI
  • Show defaults in --help
  • Add .gitignore
  • better code to employ decaying learning rate
  • Minor fixes
  • Improve readme and command line help
  • Enable execution of main Python scripts train.py and sample.py
  • Removed unnecessary line
  • Use most fequent character as prime instead of ' ' to not fail on mod…
  • select-device
  • cyrillic sampling bugfix
  • Add explanatory comments to files
  • Make sampling with a unicode prime work correctly
char-rnn-tensorflow list of languages used
More projects by sherjilozair View all
Other projects in Python