in Guides, Productivity, Resources

Torch-rnn: MacOS Installation Guide (2017 Average Joe Edition)

Disclaimer: I’m no expert in the fields of deep learning tech or AI. Just an average guy interested in learning more about what this tech can do to improve people’s’ lives!

Summary: this post will provide you a detailed, foolproof method of installing Torch-rnn so you don’t get any errors when trying to train your network with sample data. Skip down below for instructions!

It’s meant to expand/clarify on Jeff Thompson’s tutorial guide from 2016, so only follow my guide if you’re running into any installation/training issues!

Introduction

Recently, I’ve been trying to get a better idea of how digital bots (through machine learning & neural networks) can augment a user’s workflow.

Specifically, I really wanted to examine how neural networks could potentially help a person spark new ideas for a writing project, primarily by having an algorithm that gets trained (or rather learns) by input data.

Google’s Deep Dream project, for example, learns from lots of images. Based on what it learns, the neural network is able to generate some surreal images, giving us a taste of how AI can visually interpret data fed to them.

Fortunately, there are some tools publicly available for people to try building their own neural networks that can be tailored for their own projects.

I figured that, to get my feet wet, I should follow Robin Sloan’s project and build myself a neural network-enhanced suggestion tool that can help me create interesting texts geared for creative writing.

Pain Points

However, I ran into frustrating problems trying to install one of the required tools needed in Mr. Sloan’s project: Torch-rnn. It’s a neural network tool for text-based learning using single letter characters as the basis for inputting & outputting data.

I was following a tutorial guide by this cool author, Jeff Thompson to try to install Torch-rnn properly and get some sample training data started. However, I kept running into too many errors in testing my sample input due to installation files not being configured properly (it’s the fault of the command-line installers).

While Mr. Thompson’s instructions are correct, you may still run into install errors, because the installers will not automatically point to the right folder locations during their process.

The HDF5 library is usually the most common culprit that has interfered with other users’ ability to get Torch up and running.

I also ran into problems with the bundled torch-rnn/lua installations being dependent on specific versions of hdf5 (including old xCode Command Line tools).

Hopefully, this post will clear things up and not cause you any more confusion!

Pre-Instructions

Before installing Torch, we need to prepare our Macbook environment so it can play nice with Torch:

Open up Terminal.app (it’s the Mac version of Windows Command Prompt), then type/paste this line: brew install hdf5@1.8
Hit Enter.

We need Python to run our training scripts: To install Python, open Terminal.app, then type/paste: sudo easy_install pip
Hit enter. Real easy!

To make sure our Lua packages gets properly installed later on, we need Terminal to use the old XCode Command Line Tools to get the Lua packages configured with no problems.

If your Terminal.app is still open, type/paste this line:
sudo xcode-select --switch /Library/Developer/CommandLineTools
Hit enter.

If you did this correctly, Terminal should have switched to using a version 7.2 OR 7.0.2.
You can check if this is correct by typing, then hitting enter: 
clang --version

Note: The Mac App Store application may show up a new update saying your XCode is outdated. When you’re done with this tutorial, open up Terminal.app, and simply type & enter the following:sudo xcode-select -r

This tells Terminal to revert back to your original XCode Command Line Tools installation (which is likely more updated).

Alright! On to the next steps.

Primary Instructions: Torch+LUA

Now, let’s follow Jeff Thompson’s steps outlined in his tutorial.

We want to ideally install Torch in our home folder (it should look like this: “/Users/[yourusername]/torch”), which we will be doing in his tutorial.

Before following these steps, type & enter the following so you’re in your home folder:
cd ~

Now, follow these steps:

  1. Type/Paste, then enter: git clone https://github.com/torch/distro.git ~/torch --recursive
  2. Type/Paste, then enter: cd ~/torch
  3. Type/Paste, then enter: bash install-deps
  4. Type/Paste, then enter: ./install.sh

As Jeff notes, these 4 steps do the following:

This downloads the Torch repository and installs it with Lua and some core packages that are required.

Secondary Instructions: Editing .bash_profile

  • OPEN your “.bash_profile” text file using Terminal, by pasting & entering this line: touch ~/.bash_profile; open ~/.bash_profile

A text editor for .bash_profile should have opened up.

We’re going to change Mr. Thompson’s instructions by ADDING these lines to the newly opened file. Don’t forget to insert your username:

. /Users/[yourusername]/torch/install/bin/torch-activate

# TORCH
export PATH=$PATH:/Users/[yourusername]/torch/install/bin

As an example, this is what your replaced username should look like:

. /Users/JohnSmith/torch/install/bin/torch-activate

# TORCH
export PATH=$PATH:/Users/JohnSmith/torch/install/bin
  1. Save this file, but DON’T close it yet.
  2. With the text file active, click on File>Duplicate.
  3. Close your current “.bash_profile” text file.
  4. Click on Finder, then on the top of your menubar, click Go>Home.
  5. Look for “.bash_profile copy”.
  6. RENAME “.bash_profile copy” TO “.profile”

You’re basically making sure that Terminal can call up the Torch tool whenever you want. Let’s check to see if it works!

If you restart Terminal (Quit, then open up a new instance), type & enter:
th

You should see a big Torch logo and a new prompt. This means that Torch was successfully installed!

Exit Terminal again. Or, type the following to get out of the Torch prompt: os.exit()

Third Set of Instructions: HDF5 Installation

For the purposes of this tutorial, I’m skipping the GPU/Cuda instructions, because I don’t have a GPU inside my Macbook.

We’re gonna install the HDF5 library. Open up Terminal again (or if it’s already open):

  1. Type, then enter: brew tap homebrew/science
  2. Type, then enter: brew install hdf5 --with-mpi 

We’re gonna do some manual clicking here:

  1. Download the master torch-hdf5 repository ZIP folder from github. (if it’s in a .zip file, make sure to drag the “torch-hdf5 master” folder somewhere to your desktop)
  2. RENAME “torch-hdf5 master” TO “torch-hdf5”
  3. Put this “torch-hdf5” folder inside the “/Users/[yourusername]/torch” folder. You should now have a “/Users/[yourusername]/torch/torch-hdf5” directory
  4. With Terminal, type & enter: cd ~/torch/torch-hdf5
  5. Type & enter: luarocks make hdf5-0-0.rockspec

Fourth Set of Instructions: HDF5 Library for Python

Type & Enter: sudo pip install h5py

To see if this library installed successfully, type & enter each line, one by one:

python 
import h5py

It may look like nothing happened (should be no errors). That’s good! You can exit out of python by entering this line: exit()

Fifth Set of Instructions: Torch-rnn Install

With our libraries set, we can begin to install Touch-rnn!

  1. Download the “Torch-rnn master” ZIP from this github
  2. Extract the “Torch-rnn master” folder
  3. RENAME the folder to “torch-rnn”
  4. Drag this folder to the “/Users/[yourusername]/torch” folder. You should now have a “/Users/[Yourusername]/torch/torch-rnn” directory.

We’re almost to the finish line!  

Sixth Set of Instructions: HDF5 Configuration

We need to add some custom steps missing in Jeff’s tutorial.

  1. We need to OPEN Finder and click on the drive containing your Mac OS directory (should be called Mac OS for default installs).
  2. Since you can see Hidden Files, look for “usr/local/Cellar/hdf5@1.8”. Check to see if there’s a 1.8.18 folder.
  3. Open up a new Finder window. Look for “usr/local/Cellar/hdf5”
  4. COPY the 1.8.18 folder & PASTE to “usr/local/Cellar/hdf5”You should now have a “usr/local/Cellar/hdf5/1.8.18” directory.
  5. With Finder, open “config.lua” LOCATED IN “users/[yourusername]/torch/install/share/lua/5.1/hdf5” directory
  6. You should see this specific line in “config.lua”HDF5_INCLUDE_PATH = "",
  7. Between the quotation marks listed in the previous step, ADD this line: /usr/local/Cellar/hdf5/1.8.18/include
  8. Your “config.lua” file should now look something like this: 
    hdf5._config = { 
     HDF5_INCLUDE_PATH = "/usr/local/Cellar/hdf5/1.8.18/include",
    HDF5_LIBRARIES = "/usr/local/lib/libhdf5.dylib;/usr/local/lib/libsz.dylib;/usr/lib/libz.dylib;/usr/lib/libdl.dylib;/usr/lib/libm.dylib"
    }
  9. With Terminal, type & enter: brew switch hdf5 1.8.18
  10. You’re done here!

Final Set of Instructions: Prepare & Train Data

With everything checked out, you shouldn’t run into anymore problems.

Using Mr. Thompson’s final steps, we’re going to play with the same data sample listed in his guide.

  • Open Terminal.app. Type & Enter: cd ~/torch/torch-rnn

All of your inputting/outputting done in Terminal will happen in this folder. If you try to run your Python scripts outside this directory, the tool is never going to work correctly.

With Python installed, paste & enter (all one line) in Terminal:

python scripts/preprocess.py --input_txt data/tiny-shakespeare.txt --output_h5 data/tiny_shakespeare.h5 --output_json data/tiny_shakespeare.json

You should see a result that Mr. Thompson shows in his tutorial:

Total vocabulary size: 65
Total tokens in file: 1115394
Training size: 892316
Val size: 111539
Test size: 111539
Using dtype <type 'numpy.uint8'>

So far, so good.

Now, for the moment of truth. Type & Enter in Terminal (all one line):
th train.lua -input_h5 data/tiny_shakespeare.h5 -input_json data/tiny_shakespeare.json -gpu -1

If done right, you should be seeing your terminal running a lot of “EPOCH” lines, as Mr. Thompson demonstrates:

Running with CUDA on GPU 0
Epoch 1.00 / 50, i = 1 / 17800, loss = 4.163219      
Epoch 1.01 / 50, i = 2 / 17800, loss = 4.078401      
Epoch 1.01 / 50, i = 3 / 17800, loss = 3.937344
...

…If you see this result, you should now be able to follow the rest of Mr. Thompson’s steps to fine-tune the training.

Note: Notice that these Torch commands using the “th” logic have “-gpu -1” at the end. This is because these scripts are assuming that you’re using a GPU to render results.

If you’re like me and have a Macbook with no GPU, you must force the tool to calculate its results using CPU-mode by adding “-gpu -1” at the end of all of your training commands!

Now, if we do the next step (which is one of the last ones), typing & entering:
th sample.lua -checkpoint cv/checkpoint_10000.t7 -length 2000 -gpu -1

You’ll see some weird Shakespeare dialogue going on. Observe one of my own results:

Go pull ponance the lest us divers, good

He cheech a possess in your everturn.

Conclusion

How about that y’all? It ain’t fancy, but this guide should help you get started with training with other data sets.

You could use a .txt file containing Twitter feeds, Facebook posts or what have you to get a bot to chew out interesting content.

Heck, maybe you might want to throw a transcript of the Harry Potter series at Torch and see what kind of results you can come up with. The possibilities are endless!