spwnn - iteration

The fastest computer I own can validate the 49K dictionary in about 45 seconds.

The laptop computer I took with me on my programmer retreat (i7-6500U Skylake U/Y @ 2.6 GHz - 2 real cores) took 30 seconds just to do the words that start with 'a'.  In fact, my early benchmarks were based on the time it took to validate the words that started with 'a', because I didn't want to test the whole dictionary.  

Looking back at my notes I see I had a single core version in Go when I started on the trip (i.e., no goroutines) and one of the first goals was to parallelize it, which I did, and which cut test time in half on that machine.  The entire validation pass in parallel is almost three minutes.  Yech!

I never tried the 370K dictionary.  That would have taken forever!

For <reasons> I had some AWS credits burning a hole in my pocket (they expired last Dec 31, 2022).  It was time to "finish" this project!  (Nothing is ever finished, right?)  This motivated me to use the fastest AWS machine I could find for fast iteration.  That was a r6i.32xlarge which could validate the entire dictionary in 7 seconds but also cost $8.00 / hour.  This was a great boost to productivity!  99% of the time I was using one core of this beast (in VIM) but when I wanted to test, my results just popped right up in seconds!  And I could test the 370K dictionary.  So while using this expensive machine was overall inefficient it was highly effective, and effectiveness matters more!

As the end of year 2022 was approaching, I realized Graviton 3 had been released (I had still thought it was in preview).  I switched to the biggest Graviton 3 machine which is only a bit over $2.00 / hour to rent.  Those are hobbyist rates! (I mean, not if you leave it on all the time, but people spend way more on hobbies than $2.00 / hour.)

So 4x in savings right there.  And it validates the 49K dictionary in ... 6 seconds.  The 370K dictionary takes 6 1/2 minutes on the Graviton 3 64 core.

I love Graviton.  No hyperthreading!  Cheap!  Available!

Another benefit of fast iteration was I saw a pattern that was too grumpy to see while sitting around waiting for my own computer.  It's obvious in retrospect, but the way forward is to have a list of 30 words - the 12 that don't fully resolve to themselves and a two dozen more as a test case.  That test would run in not-measurable-time me thinks.  So that's on the table for the future.  Then if that test fails run the whole thing (if necessary) to figure out the issue.

The current benchmarks based on go 1.18.6 are here:  https://github.com/Above-the-Garage/spwnnmark

Gus who lives at the house where I had my programming retreat.  He doesn't like hyperthreading either.