Neanderthal’s default option is to use use native libraries, so it is very important that you do not skip any part of this guide.
How to Get Started
- Walk through this guide, set up your development environment, and try the examples.
- Familiarize yourself with Neanderthal’s more detailed tutorials and API documentation.
This is what you’ll be able to do after the installation:
uncomplicate.neanderthal.native in your namespace, and you’ll be able to call appropriate functions from the Neanderthal library.
(ns example (:use [uncomplicate.neanderthal core native]))
Now you can create neanderthal vectors and matrices, and do computations with them.
Here is how we create two double-precision vectors and compute their dot product:
(def x (dv 1 2 3)) (def y (dv 10 20 30)) (dot x y)
This is one of the ways to multiply matrices:
(def a (dge 3 2 [1 2 3 4 5 6])) (def b (dge 2 3 [10 20 30 40 50 60])) (mm a b)
Overview and Features
Neanderthal is a Clojure library for fast matrix and linear algebra computations that supports pluggable engines:
- The native engine is based on a highly optimized native Automatically Tuned Linear Algebra Software (ATLAS) library of BLAS and LAPACK computation routines.
- The GPU engine is based on OpenCL BLAS routines for even more computational power when needed. It uses ClojureCL. Check out Uncomplicate ClojureCL if you want to harness the GPU power from Clojure for your own algorithms.
- Data structures: double and single precision vector, double and single precision general dense matrix (GE);
- BLAS Level 1, 2, and 3 routines;
- Various Clojure vector and matrix functions (transpositions, submatrices etc.);
- Fast map, reduce and fold implementations for the provided structures.
On the TODO List
- LAPACK routines;
- Banded, symmetric, triangular, and sparse matrices;
- Support for complex numbers;
- Add Neanderthal jars to your classpath (from the Clojars).
- To use the native engine: install ATLAS on your system following Native Engine Requirements).
- To use the GPU engine: install the drivers and an OpenCL platform software provided by the vendor of your graphic card (you probably already have that) - (see GPU Engine Requirements).
The most straightforward way to include Neanderthal in your project is with Leiningen. Add the following dependency to your
project.clj, just like in the Hello World project:
Neanderthal’s data structures are written in Clojure, so many functions work even without native engines. However, you probably need Neanderthal because of its fast BLAS native and/or GPU engines. Here is how to make sure they are ready to use.
The native library used by Neanderthal’s native engine
Works on Linux, OS X, and Windows!
Mac OS X
Works out of the box. You should have Apple’s XCode that comes with Accelerate framework and that’s it - no need to fiddle with ATLAS, and you get Apple’s highly tuned BLAS engine.
Linux - optimized (recommended)
Neanderthal uses the native ATLAS library and expects that you make it available on your system, typically as a shared libatlas.so ATLAS is highly optimized for various architectures - if you want top performance you have to build ATLAS from the source. Do not worry, ATLAS comes with automatic autotools build script, and a detailed configuration and installation guide. Use the latest stable ATLAS, not an alpha or beta version.
If you do not follow this procedure, and use a pre-packaged ATLAS provided by your package manager (available in most distributions), you will probably get lower performance compared to a properly tuned ATLAS (but still much faster than Java).
Either way, Neanderthal does not care how ATLAS is provided, as long it is on the path an was compiled as a shared library. It can even be packed in a jar if that’s convenient, and I could make some steps in the future to make the “compile it and install ATLAS by yourself” step optional. But, I do not recommend it, other than as a convenience for people who are scared of the terminal and C tools.
This is how I installed it on Arch Linux:
- I had to have gcc (installed by default) and gcc-fortran packages.
- I followed the aforementioned atlas build instructions to the letter. The only critical addition is to add
--sharedflag (explained in the details there, but not a default).
- I had to disable Hyperthreading in BIOS (after the build, you can turn it back on)
- I had to disable CPU throttling with this command in the shell:
cpupower frequency-set -g performance(depending on the distribution and the CPU, this may be slightly different)
- I had to create a symlink
/usr/lib, that points to ‘libsatlas.so’ (serial) or ‘libtatlas.so’ (parallel) atlas shared binary created by the build script.
That should be all, but YMMV, depending on your hardware and OS installation.
Linux - non-optimized, but easy way (not recommended)
Use atlas build provided by your package manager. Something like:
sudo pacman -Suy atlas-lapack
or your distribution’s equivalent. It is fine as an easy way to get started, but does not offer full performance.
Windows - non-optimized, but easy way
Find a previously compiled libatlas.dll and put it on your path (through Control Panel’s Environment Variables or just by dropping it in the root of your project’s folder). If you can not find libatlas.dll, send me a mail to email@example.com, and I will send you mine. It would be nice if you write a few sentences about what you are planning to create with Neanderthal :)
Windows - optimized
Neanderthal uses the native ATLAS library and expects that you make it available on your system, typically as a shared libatlas.dll ATLAS is highly optimized for various architectures - if you want top performance you have to build ATLAS from the source. Do not worry, ATLAS comes with automatic autotools build script, and a detailed configuration and installation guide. Also consult the Linux section of the page you are currently reading.
Please note that the build and optimization proccess is straightforward on Linux, but needs a lot of care and patience on Windows. You’ll need to use stable ATLAS versions (I use 3.10.3) and cygwin with MinGW. Be prepared to read the ATLAS build documentation really carefully, or hire someone to help you.
GPU drivers for the GPU engine
Everything will magically work (no need to compile anything) on Nvidia, AMD, and Intel’s GPUs and CPUs as long as you have appropriate GPU drivers.
Works on Linux, Windows, and OS X!
Follow the ClojureCL getting started guide for the links for the GPU platform that you use and more detailed info.
If you use a pre-2.0 OpenCL platform (Nvidia and/or OS X), you’ll have to use
with-default-1 from the ClojureCL’s legacy namespace instead of
with-default that are used in the examples.
Where to Go Next
Hopefully this guide got you started and now you’d like to learn more. I expect to build a comprehensive base of articles and references for exploring this topic, so please check the All Guides page from time to time. Of course, you should also check the Neanderthal API for specific details, and feel free to take a gander at the source while you are there.
Tell Us What You Think!
Please take some time to tell us about your experience with the library and this site. Let us know what we should be explaining or is not clear enough. If you are willing to contribute improvements, even better!