Loading repository data…
aalmendoza / repository
Multi-layer Recurrent Neural Networks (LSTM, RNN) for token-level language models in Python using Tensorflow
Token level RNN language model for any given code corpus.
Before training the language model on a code corpus, it is necessary to tokenize the code first. Assuming that the corpus is located the directory corpus_dir and contains C code files, this can be achieved by the following
cd source
python3 utils/tokenize_corpus.py corpus_dir ".c" ../data/example/files
Doing so will store the tokenized files of the corpus in the directory ../data/example/files. Next we will need to convert this tokenized corpus into a single file that will be used as input to the language model. Following our example, this is done by
python3 utils/create_input_from_corpus.py ../data/example/files/ ".c" ../data/example/ .7 .15 .15 --vocab_size 100
Running this command will split the corpus into 70% training data, 15% validation data, and 15% testing data as well as produce the RNN LM input file for each set. In addition, the corresponding token types and the files used in each split are logged. Note to check all of the arguments by passing -h to utils/create_input_from_corpus.py. In ../data/example you will find the following generated files.
files test.txt train.txt valid.txt
rev test_types.txt train_types.txt valid_types.txt
test_files.txt train_files.txt valid_files.txt
Since we specified a vocbulary size of 100, in train.txt, valid.txt, and test.txt the top 100 most frequent tokens in the corpus will appear verbatim and all other tokens will be replaced by the <unk> token. A value of -1 for vocab_size indicates to make the vocabulary size equal to the number of unique tokens in the corpus.
Now we can train the model using the file train.txt as input. For brevity, many of the options for train.py are excluded.
python3 train.py ../data/example/ ../save/example
If we wanted to train a reverse reading language model we would instead use
python3 train.py ../data/example/rev ../save/example/rev
After training the model, we can generate code based on the language model by running
python3 sample.py ../save/example
Selected from shared topics, language and repository description—not editorial ratings.
sherjilozair /
Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow
hunkim /
Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
This project provides implementations with Keras/Tensorflow of some deep learning algorithms for Multivariate Time Series Forecasting: Transformers, Recurrent neural networks (LSTM and GRU), Convolutional neural networks, Multi-layer perceptron
campdav /
Tutorial: Multi-layer Recurrent Neural Networks (LSTM, RNN) for text models in Python using TensorFlow.
campdav /
Tutorial: Multi-layer Recurrent Neural Networks (LSTM) for text models in Python using Keras.
sfailsthy /
Multi-layer Recurrent Neural Networks (LSTM,RNN) for character-level language models in Python using Tensorflow.