I’m extremely excited about the new Unity3D Machine Learning functionality that’s being added.  Setting it up was a little painful though, so I wanted to share the steps I followed, with the specific versions that work (I tried a whole lot and nothing else worked).  In this guide, I’ll show you everything you need to get setup and ready to start with the 3D ball example.  There’s also a video version at the end..

Installing the CUDA toolkit

You’ll need to download CUDA 8.0.61 for this to work.

You can view the CUDA download archive here: https://developer.nvidia.com/cuda-toolkit-archive

Select and download CUDA Toolkit 8.0 GA2

Run the Installer

Choose Express

CUDNN Setup

Next we need to grab the CUDA Deep Neural Network library aka cuDNN

Visit the CUDNN page here: https://developer.nvidia.com/cudnn

You’ll need to create an NVIDIA account and log in to download the library.  It’s easy to do and free though.

Once you’re logged in, click the download button.

 

Choose the download link for v6.0 for CUDA 8.0 – Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0

Copying the CUDNN Files into the Toolkit Folder

Open the cuDNN zip file.

Copy the 3 folders (bin, include, and lib) from the zip file into your CUDA 8.0 folder.

The default folder path you’d copy into is C:\Program Files\NVIDIA GPU Computing Toolkit\cuda\v8.0

 

Environment Variables & Path

Now you need to add an environment variable and two path entries.

Open the environment variable dialog (hit the windows key and start typing envir…. and it’ll auto popup for you)

Click the Environment Variables Button

Click the New button to add a new System Variable

Set the Variable Name to CUDA_HOME

Set the Value to C:\Program Files\NVIDIA GPU Computing Toolkit\cuda\v8.0

Editing the Path

Find the Path in the Environment Variables Dialog

Make sure you select the System variables version, not the user variables!

Click Edit

Add the following two folders to the path.

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64

Click Ok a couple times to close out the dialogs.

 

Anaconda Setup

Next, we need to install Anaconda to configure our python environment.

Visit the download page here: https://www.anaconda.com/download/

Download the 3.6 version (I went with the 64 bit installer, not sure if it matters but I’d go with that one).

Run the Anaconda Installer and choose the default options all the way through.

 

After the installation completes, open the Anaconda Prompt

Creating the Conda Enviroment

Next, we need to create an environment with python 3.5.2.

Run the conda create command in the prompt like this:

conda create -n tensorflow-gpu python=3.5.2

Next, activate the newly created environment with this command:

activate tensorflow-gpu

And finally, install tensorflow with this command

pip install tensorflow-gpu

 

Once the installation completes, you can test that it was successful by launching python (still from that anaconda prompt) by typing:

python

Then use the command:

import tensorflow as tf

 

Downloading the Sample Project

To try out the ML agents, you’ll want to download the sample project from the GitHub page here: https://github.com/Unity-Technologies/ml-agents

You can download the zip or use GIT and clone the repository.

Next, open another anaconda prompt as an administrator.

Change directory into the ‘python’ subfolder or the location you cloned or unzipped the project into.

For example, my folder is: C:\Git\ml-agents\python because I cloned the repository into my c:\git folder.

Fixing Html5lib

Next, type the following command: conda install –force html5lib 

Now type: pip install .

Next Steps

If there are no errors, you’re good to go and ready to start setting up your Unity project with tensorflow and machine learning!

Next, you’ll want to build the Unity Environment.  The steps for this are very clearly laid out here so I won’t repeat them: https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Getting-Started-with-Balance-Ball.md

One issue I ran into though was the ENABLE_TENSORFLOW define being cleared out after I installed the tensorflowsharp library.  When this happened, the “Internal” option disappeared.  Simply re-add it and hit play so it re-compiles, then the internal option should re-appear.

Special Thanks

First I wanted to say thanks to the guys at Unity for building this all out.  I’m excited to start integrating machine learning into projects for my AI.

I also wanted to thank Nitish Mutha for this awesome blog post that got me 90% of the way through this setup.

Video