Table of contents:
To use sci as a shared library from e.g. C++, follow along with this tutorial. We illustrate
what is happening when you run the tasks libsci:compile to build the shared libsci library
and headers; and e.g. libsci:compile-cpp to build a trivial executable in a specific language, that
uses the shared libsci library to evaluate arbitrary input strings as Clojure code.
There are instructions for using the shared library from C++, or Python using ctypes, or from
Rust using bindgen.
If you want to compile libsci yourself, prepare as follows:
GRAALVM_HOME. Currently we use Oracle GraalVM 21 (double-check .github/workflows/ci.yml for what CI is currently using).Then, to use libsci from a specific language, you should have its tools, listed in each
section below (e.g. g++ to compile C++).
Convenient babashka tasks are provided to compile libsci and most of the examples mentioned
here, which can be run from the sci project root directory. e.g. bb libsci:compile.
See bb tasks for the full list of tasks.
In libsci/src we have the following Clojure file:
(ns sci.impl.libsci
(:require [cheshire.core :as cheshire]
[sci.core :as sci])
(:gen-class
:methods [^{:static true} [evalString [String] String]]))
(defn -evalString [s]
(sci/binding [sci/out *out*] ;; this enables println etc.
(str (sci/eval-string
s
;; this brings cheshire.core into sci
{:namespaces {'cheshire.core {'generate-string cheshire/generate-string}}}))))
This file is compiled into a Java class with one static method,
evalString. This will be our API for the native library. To make this library
more interesting, we enable println by providing a value for *out* in the
interpreter. Also we make the cheshire
library available, just to show that you can bring in your own Clojure
functions.
Now let’s have a look at the bridging class between Java and C++:
package sci.impl;
import org.graalvm.nativeimage.c.function.CEntryPoint;
import org.graalvm.nativeimage.c.type.CCharPointer;
import org.graalvm.nativeimage.c.type.CTypeConversion;
import org.graalvm.nativeimage.c.type.CConst;
public final class LibSci {
@CEntryPoint(name = "eval_string")
public static @CConst CCharPointer evalString(@CEntryPoint.IsolateThreadContext long isolateId, @CConst CCharPointer s) {
String expr = CTypeConversion.toJavaString(s);
String result = sci.impl.libsci.evalString(expr);
CTypeConversion.CCharPointerHolder holder = CTypeConversion.toCString(result);
CCharPointer value = holder.get();
return value;
}
}
Here we wrap the static method evalString into a native library function that
is given the name eval_string. We use GraalVM’s API to convert between Java
and C types.
The Clojure and Java code is compiled into .class files. Next, we compile those
.class files into a shared library using native-image:
$ $GRAALVM_HOME/bin/native-image \
-jar $SCI_JAR \
-cp libsci/src \
-H:Name=libsci \
--shared \
...
This begets the files graal_isolate_dynamic.h, graal_isolate.h, libsci.h,
libsci.dylib (on Linux libsci.so, on MS-Windows libsci.dll) and libsci_dynamic.h.
We move all these files to libsci/target.
In addtion, on MS-Windows, there is one more library file,
libsci.lib, which should be copied over as sci.lib.
Now, these headers and shared objects can be used natively from C++, or used to generate foreign bindings from other languages.
g++ to compile C++ code.Let’s use the library from a C++ program now. Here’s the code:
#include <iostream>
#include <libsci.h>
int main(int argc, char* argv[]) {
graal_isolate_t *isolate = NULL;
graal_isolatethread_t *thread = NULL;
if (graal_create_isolate(NULL, &isolate, &thread) != 0) {
fprintf(stderr, "initialization error\n");
return 1;
}
char *result = eval_string((long)thread, &argv[1][0]);
std::cout << result << std::endl;
return 0;
}
This code gets the first command line argument and feeds it into libsci’s
function eval_string. We compile this code as follows:
$ g++ libsci/src/from_cpp.cpp -L libsci/target -I libsci/target -lsci -o libsci/target/from_cpp
To run, we first have to set an environment variable to locate the shared libary:
$ export DYLD_LIBRARY_PATH=libsci/target
On Linux this environment variable is called LD_LIBRARY_PATH.
Now, let’s run it.
$ time libsci/target/from_cpp "
(println :foo)
(require '[cheshire.core :as cheshire])
(cheshire/generate-string {:a 1})"
:foo
{"a":1}
libsci/target/from_cpp 0.01s user 0.01s system 64% cpu 0.026 total
It worked. First we printed a keyword from within the interpreter. Then we returned a Clojure hash-map that was converted into JSON by cheshire. And then we printed the JSON string from the C++ program.
cargolibclangTo use libsci from a Rust program, we use the same shared lib generated in the previous
section (produced by running the libsci:compile task). Here we describe what happens when you
run the libsci:compile-rust task.
To build Rust language bindings to libsci, we use
bindgen which need a build.rs
file.
This file is located in libsci/from-rust/build.rs.
extern crate bindgen;
use std::env;
use std::path::PathBuf;
fn main() {
let path = env::var("LIBSCI_PATH").unwrap();
println!("cargo:rustc-link-lib=sci");
println!("cargo:rustc-link-search={path}", path = path);
let bindings = bindgen::Builder::default()
.header(format!("{path}/libsci.h", path = path))
.clang_arg(format!("-I{path}", path = path))
.generate()
.expect("Unable to generate bindings");
let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
bindings
.write_to_file(out_path.join("bindings.rs"))
.expect("Couldn't write bindings!");
}
Learn more about build.rs files here.
Secondly we write a main program that uses these bindings to call libsci. This
code is located in libsci/from-rust/src/main.rs.
#![allow(non_upper_case_globals)]
#![allow(non_camel_case_types)]
use std::ffi::{CStr, CString};
use std::str::Utf8Error;
use std::{env, ptr};
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
fn eval(expr: String) -> Result<&'static str, Utf8Error> {
unsafe {
let mut isolate: *mut graal_isolate_t = ptr::null_mut();
let mut thread: *mut graal_isolatethread_t = ptr::null_mut();
graal_create_isolate(ptr::null_mut(), &mut isolate, &mut thread);
let result = eval_string(
thread as i64,
CString::new(expr).expect("CString::new failed").as_ptr(),
);
CStr::from_ptr(result).to_str()
}
}
fn main() {
let args: Vec<String> = env::args().collect();
let result = eval(args[1].to_owned());
match result {
Ok(output) => println!("{}", output),
Err(_) => println!("Failed."),
};
}
To compile the main program, run the libsci:compile-rust task. It should create a new
libsci/target/from-rust executable.
Next, export DYLD_LIBRARY_PATH (LD_LIBRARY_PATH on Linux) to libsci/target.
Now, you should be able to run from-rust:
$ libsci/target/from-rust "(require '[cheshire.core :as json]) (json/generate-string (range 10))"
[0,1,2,3,4,5,6,7,8,9]
ctypes module.To use the shared library from Python via ctypes, do the following from the directory
containing the shared object:
$ python
Python 3.8.5 (default, Sep 5 2020, 10:50:12)
[GCC 10.2.0] on Linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> dll = CDLL("./libsci.so")
>>> isolate = c_void_p()
>>> isolatethread = c_void_p()
>>> dll.graal_create_isolate(None, byref(isolate), byref(isolatethread))
0
>>> dll.eval_string.restype = c_char_p
>>> result = dll.eval_string(isolatethread, c_char_p(bytes("(+ 1 8)", "utf8")))
>>> result
b'9'
The above instructions are for a Linux system.
For Mac OS, the file extension of the shared library should be different, probably .dylib.
For Windows, the file extension of the shared library should be different, probably .dll.
Also it may be necessary to use WinDLL instead of CDLL.
N.B. Testing has only been done on Linux.