I’m extremely excited about the new Unity3D Machine Learning functionality that’s being added. Setting it up was a little painful though, so I wanted to share the steps I followed, with the specific versions that work (I tried a whole lot and nothing else worked). In this guide, I’ll show you everything you need to get setup and ready to start with the 3D ball example. There’s also a video version at the end..
Installing the CUDA toolkit
You’ll need to download CUDA 8.0.61 for this to work.
You can view the CUDA download archive here: https://developer.nvidia.com/cuda-toolkit-archive
Select and download CUDA Toolkit 8.0 GA2
Close any open Unity and Visual Studio instances (you’ll have to restart the installer if you don’t do this first)
Run the Installer
Choose Express
CUDNN Setup
Next we need to grab the CUDA Deep Neural Network library aka cuDNN
Visit the CUDNN page here: https://developer.nvidia.com/cudnn
You’ll need to create an NVIDIA account and log in to download the library. It’s easy to do and free though.
Once you’re logged in, click the download button.
Choose the download link for v6.0 for CUDA 8.0 – Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0
Copying the CUDNN Files into the Toolkit Folder
Open the cuDNN zip file.
Copy the 3 folders (bin, include, and lib) from the zip file into your CUDA 8.0 folder.
The default folder path you’d copy into is C:\Program Files\NVIDIA GPU Computing Toolkit\cuda\v8.0
Environment Variables & Path
Now you need to add an environment variable and two path entries.
Open the environment variable dialog (hit the windows key and start typing envir…. and it’ll auto popup for you)
Click the Environment Variables Button
Click the New button to add a new System Variable
Set the Variable Name to CUDA_HOME
Set the Value to C:\Program Files\NVIDIA GPU Computing Toolkit\cuda\v8.0
Editing the Path
Find the Path in the Environment Variables Dialog
Make sure you select the System variables version, not the user variables!
Click Edit
Add the following two folders to the path.
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64
Click Ok a couple times to close out the dialogs.
Anaconda Setup
Next, we need to install Anaconda to configure our python environment.
Visit the download page here: https://www.anaconda.com/download/
Download the 3.6 version (I went with the 64 bit installer, not sure if it matters but I’d go with that one).
Run the Anaconda Installer and choose the default options all the way through.
After the installation completes, open the Anaconda Prompt
Creating the Conda Enviroment
Next, we need to create an environment with python 3.5.2.
Run the conda create command in the prompt like this:
conda create -n tensorflow-gpu python=3.5.2
Next, activate the newly created environment with this command:
activate tensorflow-gpu
And finally, install tensorflow with this command
pip install tensorflow-gpu
Once the installation completes, you can test that it was successful by launching python (still from that anaconda prompt) by typing:
python
Then use the command:
import tensorflow as tf
Downloading the Sample Project
To try out the ML agents, you’ll want to download the sample project from the GitHub page here: https://github.com/Unity-Technologies/ml-agents
You can download the zip or use GIT and clone the repository.
Next, open another anaconda prompt as an administrator.
Change directory into the ‘python’ subfolder or the location you cloned or unzipped the project into.
For example, my folder is: C:\Git\ml-agents\python because I cloned the repository into my c:\git folder.
Fixing Html5lib
Next, type the following command: conda install –force html5lib
Now type: pip install .
Make sure you include that period, it’s part of the command.
Next Steps
If there are no errors, you’re good to go and ready to start setting up your Unity project with tensorflow and machine learning!
Next, you’ll want to build the Unity Environment. The steps for this are very clearly laid out here so I won’t repeat them: https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Getting-Started-with-Balance-Ball.md
One issue I ran into though was the ENABLE_TENSORFLOW define being cleared out after I installed the tensorflowsharp library. When this happened, the “Internal” option disappeared. Simply re-add it and hit play so it re-compiles, then the internal option should re-appear.
Special Thanks
First I wanted to say thanks to the guys at Unity for building this all out. I’m excited to start integrating machine learning into projects for my AI.
I also wanted to thank Nitish Mutha for this awesome blog post that got me 90% of the way through this setup.
Video