Summer adventure games

It’s the first weekday of summer vacation, and we don’t quite have childcare worked out yet, so my poor kids are stuck in the office with me this afternoon.

Kayla trying not to crack a smile

Kayla failing to not crack a smile

A few things come to mind. First, thank goodness for the Switch. Kayla’s playing Breath of the Wild quietly. Second, yay, there’s a new Monument Valley out today! That helps a ton.

Mostly though, I think I need to load up that computer in the background with old Sierra games. One of my elementary school summers (I think it was between 4th and 5th grade?) I remember spending a bunch of time at my friend’s dad’s dental office playing King’s Quest 3 and the Colonel’s Bequest on their computer.

I need to find some good adventure games for the kids. Good ones that take hours and hours to play though. :D


Silly benchmarks

It started with me being curious: if I use an Integer in Kotlin, am I going to be paying a penalty for using a boxed primitive? So I wrote some silly benchmarks to confirm. First, in Java:

public class SumTestPrimitive {
  public static void main(String[] args) {
    long sum = 0;
    for (long i = 1; i <= Integer.MAX_VALUE; i++) {
      sum += i;
    }
    
    System.out.println(sum);
  }
}

public class SumTestBoxed {
  public static void main(String[] args) {
    Long sum = 0L;
    for (Long i = 1L; i <= Integer.MAX_VALUE; i++) {
      sum += i;
    }
    
    System.out.println(sum);
  }
}

On my system (a mid-2011 MacBook Pro, 2.4 GHz i5) the primitive version takes 1.77 seconds, and the boxed version takes 20.8 seconds. Then in Kotlin:

fun main(args: Array<String>) {
	var sum = 0L
	var i = 0L
	while (i <= Integer.MAX_VALUE) {
		sum += i
		i += 1L
	}

	println(sum)
}

The Kotlin one takes 1.81 seconds. Tiny bit slower than the Java primitive one, but that’s probably just due to needing a little more time for Kotlin’s runtime to load. Kotlin does unboxed primitives properly, yay!

Now I’m curious though: how do the other languages I use on the regular perform? Let’s try Clojure first, both a straightforward implementation and one tailored to match the Java one better:

(defn sum-test-straightforward []
  (loop [sum 0
         i 0]
    (if (<= i Integer/MAX_VALUE)
      (recur (+ sum i) (+ i 1))
      sum)))

(defn ^long sum-test-gofast []
  (loop [sum 0
         i 0]
    (if (<= i Integer/MAX_VALUE)
      (recur (unchecked-add sum i) (unchecked-add i 1))
      sum)))

sum-test-straightforward took 5.1 seconds, and sum-test-gofast 1.69 seconds. The gofast one is comparable to the Java one, probably a little slower: I ran these at a REPL, so there’s no startup time involved.

Ok, how about Common Lisp? I can think of 3 approaches to take off the top of my head.

;; 2147483647 is the same value as Java's Integer/MAX_VALUE.

(defun sum-test-iterative ()
  (let ((sum 0))
    (dotimes (i 2147483647)      
      (setf sum (+ sum i)))
    sum))

(defun sum-test-recursive ()
  (labels ((sum-fn (sum i)
             (if (<= i 2147483647)
                 (sum-fn (+ sum i) (+ 1 i))
                 sum)))
    (sum-fn 0 0)))

(defun sum-test-loop ()
  (loop for i from 1 to 2147483647 sum i))

Using ClozureCL, all 3 of these perform abysmally:

  • sum-test-iterative: 62.98 seconds, 2.82 of which were GC time. 20 GiB allocated.
  • sum-test-recursive: 74.11 seconds, 3.52 of which were GC. 20 GiB allocated.
  • sum-test-loop: 50.7 seconds, 2.58 of which were GC. 20 GiB allocated.

SBCL does much better off the bat, but still not great:

  • sum-test-iterative: 7.7 seconds, no allocation
  • sum-test-recursive: 17.1 seconds, no allocation
  • sum-test-loop: 7.63 seconds, no allocation

Adding some type annotations and optimize flags helped SBCL, but ClozureCL’s times stayed the same:

(defun sum-test-iterative ()
  (declare (optimize speed (safety 0)))
  (let ((sum 0))
    (declare ((signed-byte 64) sum))
    (dotimes (i 2147483647)
      (setf sum (the (signed-byte 64) (+ sum i))))
    sum))

SBCL’s sum-test-iterative drops down to 3.13 seconds, still no allocation. No change on Clozure. I’m probably doing something wrong here, but it’s not clear to me what. The disassembly of sum-test-iterative on SBCL shows that there’s still an allocation going on there: maybe the problem is just that 64-bit integers don’t work unboxed due to SBCL’s pointer tagging?

* (disassemble 'sum-test-iterative)

; disassembly for SUM-TEST-ITERATIVE
; Size: 69 bytes. Origin: #x1002AF97D7
; 7D7:       31C9             XOR ECX, ECX                    ; no-arg-parsing entry point
; 7D9:       31C0             XOR EAX, EAX
; 7DB:       EB2D             JMP L3
; 7DD:       0F1F00           NOP
; 7E0: L0:   488BD1           MOV RDX, RCX
; 7E3:       48D1F9           SAR RCX, 1
; 7E6:       7304             JNB L1
; 7E8:       488B4AF9         MOV RCX, [RDX-7]
; 7EC: L1:   488BD0           MOV RDX, RAX
; 7EF:       48D1FA           SAR RDX, 1
; 7F2:       4801D1           ADD RCX, RDX
; 7F5:       48D1E1           SHL RCX, 1
; 7F8:       710C             JNO L2
; 7FA:       48D1D9           RCR RCX, 1
; 7FD:       41BB00070020     MOV R11D, #x20000700            ; ALLOC-SIGNED-BIGNUM-IN-RCX
; 803:       41FFD3           CALL R11
; 806: L2:   4883C002         ADD RAX, 2
; 80A: L3:   483B057FFFFFFF   CMP RAX, [RIP-129]              ; [#x1002AF9790] = FFFFFFFE
; 811:       7CCD             JL L0
; 813:       488BD1           MOV RDX, RCX
; 816:       488BE5           MOV RSP, RBP
; 819:       F8               CLC
; 81A:       5D               POP RBP
; 81B:       C3               RET
NIL

Next up, Swift:

var sum: UInt64 = 0
var count: UInt64 = 2147483647

for i: UInt64 in 0 ..< count {
    sum = sum + i
}

print(sum)

Without optimizations, 16 minutes 50 seconds. Holy shit.

With optimizations, 1.11 seconds.

Ok, last one, C:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
	long sum = 0;
	const long count = strtol(argv[1], NULL, 10);
	for (long i = 0; i <= count; i++) {
		sum += i;
	}
	printf("%ld", sum);
	return 0;
}

Why take in the count parameter from the command line? Because Clang cheats. If I use the constant in there, it’s smart enough to just precalculate the whole thing and just return the final result.

Without optimizations:

solace:sum-tests jfischer$ clang -o sum-test SumTest.c 
solace:sum-tests jfischer$ time ./sum-test 2147483647
2305843008139952128
real	0m8.247s
user	0m8.190s
sys	0m0.035s

With optimizations:

solace:sum-tests jfischer$ clang -Os -o sum-test SumTest.c
solace:sum-tests jfischer$ time ./sum-test 2147483647
2305843008139952128
real	0m0.006s
user	0m0.002s
sys	0m0.002s

It turns out, Clang still cheats even if the loop counter comes from outside. I’m pretty sure it recognized what I’m doing and just turned that loop into Gauss’ trick for computing an arithmetic series. It doesn’t matter what loop count I give it, it always takes the same amount of time with optimizations.

I can’t read/write assembly, but playing around on godbolt.org makes it look like that’s the case: https://godbolt.org/g/FmL66q. (There’s no loop in the disassembly.) And I can’t figure out how to trick it into not doing that, so I’ll call it quits for now.


Sneaking Clojure in - Part 2

I ended up turning to a Clojure REPL to solve an issue in that project I totally didn’t sneak Clojure into before and realized I did some things the hard way last time.

First up: you don’t need to create and compile a Java class from Clojure to call into Clojure code from Java. If I had actually read the Java Interop reference guide on Clojure.org, I would have noticed that there’s a section on calling Clojure from Java. It’s much, much easier.

If I define this namespace/function:

(ns project.util)

(defn get-my-thing []
  {:my :thing})

I can call it like so:

// In Java code:

// First, find the require function. Then use it to load the project.util namespace
IFn require = Clojure.var("clojure.core", "require");
require.invoke(Clojure.read("project.util"));

// After project.util is loaded, we can look up the function and call it directly.
IFn getMyThing = Clojure.var("project.util", "get-my-thing");
getMyThing.invoke();

Easy peasy. I don’t have to jump through the gen-class hoops, and bonus! I don’t have to compile my Clojure code ahead of time. I just need to make sure the source files are on the class path.

You should of course compile your Clojure code if you’re distributing an application built on it. It’ll load faster, plus you might not want it readable.

What I specifically didn’t want to hook into that project that I totally wasn’t sneaking Clojure into is a REPL: I want to be able to poke directly at the application’s state while it’s running. To do that, I’ll need to make sure that tools.nrepl is available on the classpath, and require/launch it from within the application.

I could probably use Clojure 1.8’s socket server repl instead, but I plan on using Cider to talk to it, so nrepl’s a better choice.

In Java code:

public static void launchNrepl(int port) {
  try {
    IFn require = Clojure.var("clojure.core", "require");
    require.invoke(Clojure.read("clojure.tools.nrepl.server"));

    // Note: passing "::" as the :bind parameter makes this listen on all interfaces.
    // You might not want that.
    IFn startServer = Clojure.var("clojure.tools.nrepl.server", "start-server");
    startServer.invoke(Clojure.read(":bind"), "::", Clojure.read(":port"), port);
  }
  catch (Exception e) {
    // log the error
  }
}

In my theoretical project where I totally didn’t do this I also load in a namespace of helper code I’ve written to wrap around the Java objects we already have written.


Which of Apple's game technologies can I use?

Writing this down here because I didn’t see an easy reference to it anywhere and I just took the time to figure it out.

I’m starting a new game-ish thing and figuring out what I want to build it in. What technologies can I use? In particular, I’m looking at:

I personally still own:

  • an iPad 2 running iOS 9 (oh how I wish I could roll that back to iOS 6)
  • an iPhone 4 running iOS 7
  • my daily iPhone 6 running iOS 10
  • and at work I have an iPad 1 running iOS 5 (so much nicer to use than the iPad 2).

My iMac and MacBook Pro are both on El Capitan. The iMac’s too old for Sierra, and I don’t want my daily computers on different operating systems. As far as Apple’s game related technologies go, El Capitan is the equivalent of iOS 9.

Why go as far back as that iPhone 4? My 6 year old son likes to use it. I could also suck it up and start doing Android dev and support his Kindle, but… eh.

So here’s a quick breakdown of what’s supported on those devices.

iOS 5

No Swift, no SpriteKit, no SceneKit, no GameplayKit. GLKit’s good here. I’ll need to use Xcode 6.4 to compile for it. Apple’s GLEssentials example works just fine on that iPad 1, although I’m only hitting about 21 frames per second on it, even when compiled for release.

iOS 7

Swift up to version 2.3, I think. It doesn’t look like I can still target iOS 7 using Xcode 8, so I’m limited to what’s supported in Xcode 7.3.1. No SceneKit or GameplayKit. GLKit works fine, of course.

SpriteKit’s initial release is supported, which means:

  • The scene graph
  • Physics
  • Actions
  • Texture atlases
  • Core Image filters
  • Particle systems (with editor support)
  • Really basic editor in Xcode

iOS 8

I don’t have a device that’s limited to or running this version, but I’ll include it for completeness. Xcode 8.2.1 still supports iOS 8, so Swift 3 is an option. It also introduced SceneKit, so there’s a 3D engine in there. I don’t know anything about SceneKit beyond that it does 3D, so I can’t speak more about that. (I sure as heck don’t have the time for 3D at the moment. :D)

Still no GameplayKit here. New stuff in SpriteKit:

iOS 9

GameplayKit finally comes into play here, giving us some handy tools:

Most of GameplayKit is straightforward to write yourself, but it’s nice to have well-tested, proven implementations of that stuff.

SpriteKit adds:

iOS 10

GameplayKit adds:

GameplayKit also gets some integration with SpriteKit:

SpriteKit adds:

I’ve probably missed some stuff up there, but this is the stuff I’m likely to care about.

Part of me’s tempted to go the raw OpenGL (or at least GLKit) route: gives me the most control, and I can add whatever I feel is missing, etc. It’s a huge time sink though, and at best I only have about an hour a day for personal projects right now, so it makes more sense to lean on these things when they’re available.

I’m kind of iffy about using Swift: I’d like to use it, but if I want to target my iPhone 4 I have to use Swift 2.3, and it feels silly to start a new project on what’s already an obsolete version.

I guess that leaves me two real options:

  • Support the iPhone 7, use the original SpriteKit release and Objective-C.
  • Target iOS 9, use SpriteKit and GameplayKit with Swift 3.

Sneaking Clojure into an older Java project

Now I’m not saying I did this, but if I was going to do it and took notes for future reference, this is how I’d sneak some Clojure code into an older, Ant based Java project.

First, figuring out how to just compile any Clojure code without Leiningen or Boot. You’ll need the Clojure jar, so download that from clojure.org first. The magic incantation to compile Clojure code manually is:

$ java -cp clojure-1.8.0.jar:<src-dir>:<out-dir> -Dclojure.compile.path=<out-dir> clojure.lang.Compile your.namespace

Important: both your source directory and your build output directory need to be on the classpath, and you need to tell Clojure’s compiler where to stick your output files with -Dclojure.compile.path. Also, you don’t give it a specific file to build, you tell it what namespace you want to build.

So say your source code is in src, you want to stick your compiled classes in out, and the root namespace you want to compile is blah.core. Your command line will be:

$ java -cp clojure-1.8.0.jar:src:out -Dclojure.compile.path=out clojure.lang.Compile blah.core

Next then, let’s get it into Ant. This particular project that I totally didn’t sneak Clojure into resolves most of its dependencies via Ivy, but we vendor in a few things (Ivy just doesn’t seem to pull in native libraries cleanly, for example), so I went the easy route and just added the Clojure jar to our vendored library directory. (Except I totally didn’t. No way.) The Ant target ended up looking like this:

<!-- Compile the Clojure bits of the project. -->
<target name="clojurec" depends="javac">
  <java failonerror="true" fork="true" classname="clojure.lang.Compile">
    <classpath>
      <path refid="build.class.path"/>
      <pathelement path="${out.classes.dir}"/>
      <pathelement location="lib/clojure-1.8.0.jar" />
      <pathelement path="src/clj" />
    </classpath>
    
    <jvmarg value="-Dclojure.compile.path=${out.classes.dir}" />
    <arg value="namespace.core" />
  </java>
</target>

Pretty straightforward. It’ll be relying on existing Java code, so we run javac first. Output goes into the same directory as the Java classes (defined elsewhere as out.classes.dir), and both that directory and the source directory are added to the classpath.

Except! I just noticed that even though we’re ahead-of-time compiling this code, Clojure’s compiler is also copying my source files over into the output directory. I don’t see a way to skip that, so we’ll probably just need to filter out the source directories when creating artifacts.

Anyway, how do we use that Clojure code? At the moment, I think we’re stuck using Reflection from Java code to find and call it. Let’s generate a simple class from Clojure to make sure we can call things correctly.

(gen-class
 :name myproject.ClojureTest
 :prefix clojureTest-
 :methods [[sayHello [] void]])

(defn clojureTest-sayHello [this]
  (println "Hello from Clojure land."))

Compile that with ant clojurec, then take a look at the output class file. Looks good!

$ javap build/classes/myproject/ClojureTest.class 
public class myproject.ClojureTest {
  public static {};
  public myproject.ClojureTest();
  public boolean equals(java.lang.Object);
  public java.lang.String toString();
  public int hashCode();
  public java.lang.Object clone();
  public void sayHello();
}

We have our class file, and it has the sayHello method. Let’s see if we can call it from Java land (obviously leaving out some error handling):

private void clojureTest()
{
  try
  {
    Class c = Class.forName("myproject.ClojureTest");
    Object o = c.newInstance();
    Method m = o.getClass().getMethod("sayHello", null);
    m.invoke(o, null);
  }
  catch (Exception e)
  {
    e.printStackTrace();
  }
}

And when I compile and run, I see this little bit hidden among all of our logging spew:

 [java] Hello from Clojure land.

Yay!

Bonus Round!

We use Guava in this project that I’m totally not monkeying with right now, and this Clojure code that of course I’m not adding is going to need to participate in Guava’s Event Bus. How do we do that?

Something like this:

(ns eventtest.core
  (:import [com.google.common.eventbus EventBus Subscribe]))

(defrecord AnEvent [])
(defrecord AnotherKindOfEvent [])

(definterface MyDummyInterface
  (^{Subscribe []} handleAnEvent [^eventtest.core.AnEvent evt])
  (^{Subscribe []} handleAnotherEvent [^eventtest.core.AnotherKindOfEvent evt]))

(defrecord MyListener []
  MyDummyInterface
  (handleAnEvent [this evt]
    (println "Hey, I got an event."))

  (handleAnotherEvent [this evt]
    (println "Hey, I got another event.")))
  • Import the EventBus and Subscribe classes from Guava.
  • Define an interface with a method for responding to each type of event you’re going to be listening to. It needs to be an interface (and not a protocol) for the type signatures to work, and Guava needs those type signatures.
  • Define your listener record and have it implement that interface.
  • Either your interface methods or your implementation records need to be annotated with Subscribe, like this: (^{Subscribe []} fn-name [args]). I chose to do it in the interface definition because it seems less noisy there.

Now your Clojure code can participate in the event bus without any trouble.

;; Create an event bus
(def bus (EventBus.))

;; Create a listener record
(def listener (MyListener.))

;; Register the listener with the bus. This will hook up those interface
;;  methods for you
(.register bus listener)

;; Post events to the bus, and the proper methods on that listener
;; record will be called.
(.post bus (AnEvent.))
(.post bus (AnotherKindOfEvent.))