f64ad

Introduction

This crate brings easy to use, efficient, and highly flexible automatic differentiation to the Rust programming language. Utilizing Rust's extensive operator overloading and expressive Enum features, f64ad can be thought of as a drop-in replacement for f64 that affords forward mode or backwards mode automatic differentiation on any downstream computation in Rust.

Key features

  • f64ad supports reverse mode or forward mode automatic differentiation
  • f64ad uses polymorphism such that any f64ad object can either be considered a derivative tracking variable or a standard f64 with very little overhead depending on your current use case. Thus, it is reasonable to replace almost all uses of f64 with f64ad, and in return, you'll be able
    to "turn on" derivatives with respect to these values whenever you need them.
  • The f64ad Enum type implements several useful traits that allow it to operate almost exactly as a standard f64. For example, it even implements the RealField and ComplexField traits, meaning it can be used in any nalgebra or ndarray computations.
  • Certain functions can be pre-computed and locked to boost performance at run-time.

Crate structure

This crate is a cargo workspace with two member crates: (1) f64ad_core; and (2) f64ad_core_derive. All core implementations for f64ad can be found in f64ad_core. The f64ad_core_derive is currently a placeholder and will be used for procedural macro implementations.

Implementation note on unsafe code

This crate uses unsafe implementations under the hood. I tried to avoid using unsafe, but I deemed it necessary in this case. Without using unsafe code, I had two other options: (1) f64ad_var could've used a smart pointer that cannot implement the Copy trait and, thus, f64ad could not be Copy either. This would mean that f64ad could not be an easy drop-in replacement for f64 as annoying .clone() functions would have to be littered everywhere; or (2) f64ad_var could've used a reference to some smart pointer, such as &RefCell. However, certain libraries, such as nalgebra, require their generic inputs to be static, meaning the reference would've had to be &'static RefCell in f64ad_var. In turn, ComputationGraph would also have to be static, meaning it would essentially have to be a mutable global static variable that would involve unsafe code anyway. In the end, I viewed an unsafe raw pointer to a ComputationGraph as the "best" among three sub-optimal options.

I was very careful with my internal implementations given the unsafe nature of the computations; however, PLEASE be aware that the computation graph should NEVER GO OUT OF SCOPE IF ANY OF ITS VARIABLES ARE STILL IN USE.