spwnn - the fundamental question

The question I wanted to answer for myself is why we clearly can read words "all at once" at a glance, and yet the order of the letters in the word matters.  That seems perhaps like a silly question ... but what would be an algorithm that would run in parallel on parts of the word, completely independently, and yet somehow deal with the implied word ordering.  I hypothesized that by grouping the letters of the words into pairs there would be a way to derive the original word - even if it were misspelled.

And it works well, even though there is no inherent reason it should.  One would expect it would require a deep neural network, but this is insanely shallow.

In my big dictionary of 370,000 or so words, only 255 don't return themselves as the correct word.  In my 49000 or so word dictionary only 10 or 12 don't pass that test.

That's amazing!

It supports my hypothesis that people process words in letter pairs, in parallel, at a glance.

spwnn - 40 years in the making!

I worked on this program for 40 years and I finally published it on GitHub.

It's a spelling corrector.

https://github.com/Above-the-Garage/spwnn/blob/main/README.md

I'll write more about it as I work out how to explain it.

You can try it here:

https://above-the-garage.com/spwnn

Just type in a word (misspelled or otherwise) and see what it comes up with.

I've found it useful when I am so far off on the spelling of a word that regular spelling correctors won't even give me a hint.

I hope I'm in the running for fewest lines of code produced over the longest period of time.

What model?

DeepLens is a hobby IoT device Amazon sells.  It is a small Intel-Atom powered computer (2 cores, 1.3 GHz) with a camera builtin.

I have the 1.0 version.  There is a 1.1 version that is faster, and I can see why they made that!

My version is painfully slow to use.

My original idea (just over four years ago) was to set it up and then SSH into it and poke at it from time-to-time.

But ... it's really good to have a screen so you can see what the camera is seeing!

(There is no doubt a way to do this remotely but I didn't figure it out.)

Well, I powered it up today, with the goal of doing a factory reset.  The instructions are long and complicated, basically resulting in a 16 GB USB stick that it will boot from and erase itself.  And I made it harder by doing it on the device itself!

I learned stuff about Linux file systems.  That's good.

When I was done I had evolved my setup quite a bit, from just the device plugged into an HDMI monitor and keyboard and mouse, to lots of additional power and a USB-extender.

Frankensetup for factory reset!

Luckily the Seahawks made it to the playoffs today (sadly, they failed on their quest, but dang, what a successful first half!).  So when I started up long-running operations on DeepLens I was actually busy watching American football.

Picture copied from AWS site.  Should be linked.

I remember four years ago I followed the instructions, registered the device to an AWS account, got all the IAM settings correct, and then started to follow the tutorial for doing some machine learning.

Then I hit the showstopper: "Pick your model."

What model is that?  For what?  Don't I just give it a bunch of pictures and train it?  You mean I have to understand what a model is!  Well, I don't.

Yesterday, I used a Google service to translate some speech to text.  It said, "Pick your model", but luckily it had a dropdown, so I just picked one.  

Pick your model.  Isn't that a huge part of being a data scientist who specializes in machine learning?  Knowing the strengths and weaknesses of models?

I dunno.  Someday I'll learn what a model is, and how that is different from the data.  Or I'll learn something completely different because I don't understand any of these terms at all.

It's on my list of things to learn about, this mysterious machine learning, that as I write this is generating art, text, and even computer code.

But not right now.

CellMapper

The final leg of my triangular approach to improving the quality and reliability of our internet was moving off Verizon into Google Fi.

We were on Verizon because an important part of the extended family was on it and calling within the network was free.  I mean, like 10 years ago.  And 10 years ago Verizon probably worked a bit better in our neighborhood (2G?).  But over time it got worse and worse until it would take minutes to get a text message.  Luckily with Wi-Fi calling it was not so much of a problem - except then Comcast went bonkers and we couldn't depend on Wi-Fi calling either!

Ok.  So big deal - switch providers - how hard can it be?  Well, since I also had the T-Mobile hotspot (later replaced with Starlink) I knew that T-Mobile coverage worked, albeit poorly.  But more research was required!

Enter:  CellMapper.

T-Mobile (which is what Google Fi uses in our state) had way more cell towers than Verizon.  They are also 5G towers - that's why there are so many.  So even though we are in a cell-shadow even with T-Mobile, it's not as bad.

I couldn't really test T-Mobile without getting a T-Mobile phone (because had locked Verizon phones) so it was a risk to switch carriers, but when bad is bad (our Verizon coverage), how much worse could it get?

(My wife says:  "Never challenge worse.")

Google Fi is also 50% of the cost of Verizon!  That's nice.

Anyway, moral of the story is:  Check out CellMapper!  It's awesome.

Also, speaking of legs and triangles:

BTW - if you switch cell providers for you and your family, move your phone LAST.  I thought I would be a helpful guinea pig and move my line first - but my line is the billing line!  Once I moved it, it was much harder to login to the Verizon site!  So do yourself a favor and move your line last.

The Speed

My original hypothesis was slowing down the Xfinity modem would make less use of the higher frequencies (which are more subject to interference) and my problems would be solved.

As it is, I think, from something one of the techs told me, that slower modems are just configured to run more slowly, and still use all the frequencies.

But!  I wanted to get a speed reduction from Comcast because my modem used to work before we bumped it up to gigabit.  It wasn't a pretty process.  They kept saying they had me on a new, cheaper, and slower plan, but it didn't work, and there was no confirmation email.

Once I mentioned Starlink I got a real tech on the line who configured the modem (and my billing plan) to slower (and cheaper).

I bought the slowest plan they had - 300 megabits - and then a few months later they cranked it up to 400 megabits for the same price!  So I'm at 400 megabits, which usually will go up to 440.

I suspect since the bad connector was fixed that I could go back to gigabit ... just not feeling like taking the risk right now.

One benefit of getting the slower plan (because Comcast screwed up) was my premium channels went away.  So Comcast cut the cord for me.

Thanks Comcast!

[2023-11-14 - back up to a gigabit and things are working!  My previous poor performance was caused by a single bad connector that was out of site in a bush.]

The Modem

As I started to concentrate on my Comcast issue I felt like all of my problems started when I upgraded to Gigabit service.  Previously it was ... 500 megabits?  I don't remember.

One thing about faster modems is they use higher frequency bands to push more data down the wire.  I thought, hey, maybe, for whatever reason, the infrastructure, somewhere along the way, can't handle the increased frequencies.  (Turns out I was (almost) right - it was a connector that couldn't reliably handle the higher frequencies.)

So I thought, a relatively cheap experiment would be to get a cable modem that has limited speed - one that can't do gigabit speeds.

I bought a Netgear CM1150V for a hundred bucks.  It was only rated to 400 megabits by Comcast, but it turns out it could easily handle gigabit speeds.  Oh, and it had to handle VOIP, which the CM1150V did fine with.

Well, shit.  It's still fast.  For some reason I did a web search for CM1150V, probably looking for the manual, and discovered this awesomeness:  A script for scraping the error rates of every frequency band and plotting it!

I downloaded it and I was off to the races.  

Here's a graph of the data it collected over a few days:

That's a lot of errors!

So my hypothesis about the errors being worse at higher frequencies - wrong.  But boy that's a lot of errors.  And they were worse in the morning for unknown reasons.

Now somewhere I got the idea, I think from the author of the modem scraper, Harold Holm, that if the modem gets too many errors it reboots!  So that's what was happening.  Which explains why the big outages lasted as long as it typically would take me to reboot it manually.

I get the logic of defensive programming on the cable modem - hey, if there are too many errors, maybe rebooting will fix it! But even with all those errors it worked - until the modem itself decided there were too many errors and rebooted.

Oh my.

I'm back on the Comcast modem, because I wanted to have all their own parts online when the tech came out to analyze my system.  And it turns out that for not much money you rent the modem and also get unlimited internet.  So I need to keep the modem anyway.  

Ideally I would hook up the Netgear modem again, and scrape it for more data, and see how I'm doing on errors, but since it's all working, why bother?

Comcast - one tiny error

After two years of my family and I getting five-minute internet outages once or two (or three times) a day, I finally decided to put the energy into fixing this problem.

I decided to have Comcast come out until it was fixed.  What happened twice in the preceding two years was a technician would come out, replace a lot of connectors, maybe put in a filter that made things worse, put in a signal booster, measure things, declare it fixed, and leave.  Generally the internet service was great except for these outages.

And now, in July 2022, having made up my mind to just keep calling Comcast until it was fixed ... it was fixed on the first visit.

I believe I mentioned to the Comcast phone rep that I had Starlink as my backup.  Suddenly several people at Comcast were interested in my case and talked to me on the phone.  I talked to a more senior person, and he told me, don't hold back, show the tech everything, walk the neighborhood, whatever.  Normally I feel a little bad telling someone how to do their business, but now I was empowered!

The tech that came out did a terrific job of replacing connectors and simplifying the setup - my MOCA network worked again.  He measured things and showed the measurements to me.  Everything looked great just as it always had.  We walked over to the distribution point down the street a bit, and which doesn't fully close anymore, and is kind of filled with dirt, and talked about how rain affects it.  The tech said he had seen worse and it wasn't the problem.  We went across the street to the look at the new fiber drop, which is a disaster:

and

The tech was pretty sure that wasn't Comcast fiber.  That's another story.

He cleaned up the junction box at the house:

Before

After

That's much nicer than the mess that was in there before.

But I thought ... self, you are empowered.  Maybe ... poke around a bit.

So I traced the cable from that box to where it goes into a big, beautiful conduit that goes under the neighbor's yard and driveway.  The origin of that is another story.

And as I looked ... what to my wondering eyes did appear but another connector that had been there for years!

It had been hidden in these bushes.  I pointed it out to the tech and mentioned that it didn't look like the same waterproofing he had been doing to the other connectors.  He opened it up and said, "Eww."

It was full of ... green stuff.

He replaced it, waterproofed it, and now everything is great.  One frickin' connector.  Auggh!

As to the specific cause of the five-minute outages ...

That is also another story.

Backup?

Why do I have a backup internet?

When we moved into our house way back in 1995 I had a permanent dial up connection to WorldCom or some such.  It was not supposed to be a permanent connection but nobody was using the local connection point too much, so the vendor didn't care, and so I was basically dialed up 24/7, and using Windows connection sharing to get the internet through the rest of the house.

Then one day enough people were using their service so they suddenly cared and told me I couldn't be dialed up 24/7.  Luckily just then DSL service came to our neighborhood.  But just barely as we were at the outer reaches of how far those telephone lines could carry a signal. But it worked and we went from sort-of 56K on the modem to an actual 64K.  Woohoo!

Then our small local cable company laid fiber in our neighborhood - not "FTTH" (Fiber To The Home) but fiber to distribution points that then came into our home over the cable lines.

Then Comcast bought that company many moons ago and we've had Comcast every since.

Comcast gets a bad rap but they've been pretty great to us:  generally increasing our speeds for less money - what's better than that?

Shortly before the Covid-times I upgraded to Gigabit Comcast and was happy.

But then the Covid-times hit and I was working entirely from home and I became aware that my Comcast connection would go out to lunch for about five minutes once or twice a day.  

During the first two years of Covid I did try to get it fixed.  First, I switched out the modem in case it was the problem (it wasn't); and then I called Comcast, twice during those two years, and a technicians would come out and rotate all the connectors and try some signal boosters and such.  But none of that fixed it.  It would work really well - very fast - faster than promised - but go out to lunch for five minutes.

Those big thick red lines are five minute outages.  Spoiler alert: it's the same amount of time it takes ro reboot the router.

How it looks now.  Beautiful!

It was not good for me to disappear for five minutes in the middle of an online meeting, so I needed a backup.  I went with T-Mobile home internet which was LTE and about 20 megabits down and 2-3 megabits up - but reliable, except during power outages.

A couple of years went by and I finally decided I really needed to get my Comcast connection fixed - it felt like some distribution equipment was broken in the neighborhood which caused the five minute outages.  

And while I was at it, I wanted to make my backup internet systems more reliable too.

Oh, and we got virtually no signal from Verizon, our cell carrier, and most of our phone calls were carried over Comcast via Wi-Fi calling.  So I couldn't use our cell phones data as a backup.

Things were worst during a power outage - Comcast crapped out even if I had my modem on a UPS; and T-Mobile would drop it's connection although it was on a UPS; and Verizon never worked unless we physically drove out of our cell coverage shadow (never mind about a power outage).

My plan to get Comcast fixed was to call them and have them come out again and again until they figured it out.  If things went like they had in the past, a technician would come out, change the connectors, try a new amplifier or two, and one time they swapped out the modem.  Nothing got better., so I assume going forward, a tech would come out, do the same things, and it wouldn't get better, but eventually they would have to escalate the problem inside Comcast.  That was the plan at any rate.

My plan for T-Mobile internet was to switch to Starlink after my friend HB demonstrated how far along they were with satellite deployments.

And, ironically, my plan for Verizon was to switch to Google Fi, which is T-Mobile in our area.  We get a little data over T-Mobile, but I doubt it will work during a power outage.  We haven't had an outage yet long enough to test my hypothesis.  It's got a better chance than Verizon.

It took six months to do these things and now they are all done!  I have slower but more reliable Comcast (400 megabits down); T-Mobile cell service which works (mostly) at our house; and Starlink.

In succeeding posts I'll describe what went right and wrong with my plan.  It was an eventful transition.

Starlink

Starlink is amazing.  My results are highly variable, with 150 mb down at the higher end, and 20 mb down at the lower end.  I think it's more affected by general usage in the area (since it's a shared resource) as opposed to visibility of satellites but I don't really know.  It's my backup internet.

Think of all the engineering that went into this thing:  reusable rockets; mass production of satellites that can maneuver themselves into position and have highly accurate phased array antennas; and my little dish, also with a phased array antenna that shoots a signal 350 miles into space!  That's twice the distance between Seattle, WA and Portland, OR.

I climbed up on the roof to install Dishy (as it is known) temporarily.

Later, thank goodness, I had it professionally mounted by the same person that installed our roof.

So far it's come through all the interesting weather we've had with flying colors.

If this blog post triggered you because you're mad at Elon, take comfort that SpaceX is run day-to-day by the COO Gwynne Shotwell who appears to be a genius, given the scope of SpaceX operations.

A colleague at work HB put me onto this real-time map of the 3,000 (-ish, as I write this) satellites is what convinced me to take a chance on Starlink.  

We live in an electromagnetic shadow, so cell towers don't work so great, like, at all, well, barely, so my T-Mobile backup internet kind of worked (and got my son through eight weeks of a remote trial where he was a juror), but it never worked during a power outage.  I suspect that cell towers go into a low power mode during an outage so they can provide minimal service for longer.

For Starlink, I'm genuinely lucky.  I wasn't supposed to get a dish until sometime in 2023 and it arrived in 2022.  And my base station ("internet gateway") is SpaceX satellite headquarters nearby in Redmond!  The Redmond office has ten or twelve radomes on the roof which I believe is ... a lot.

Underground!

Speaking of windstorms ...

Let me recommend the "Weather Underground" site.

It lets you see the current stats of your local weather station.  Here's our local weather station:

https://www.wunderground.com/dashboard/pws/KWASAMMA54

Here's our windstorm from yesterday, petering out this morning:

You can (in the plot) see wind gusts up to 40 mph and a general trend of wind from 15 mph trending down to zero.

What's also great is you can get information about yesterday - like, how much did it rain?  Or how fast was the wind yesterday?

Windy

There's a bit of a windstorm going on as I type this.

Which reminded me of this awesomeness:

Wind Visualization

The link is centered around Puget Sound, but you can scroll around to see anywhere on Earth.

Guild Wars 2 - Best Business Model

Certainly (I'm prejudiced) Guild Wars 2 is an incredible value for a small amount of $$ !

"Brianna Royce: Guild Wars 2. This has been my nomination for the last many years now, and it’s still true: There is no MMORPG on the market right now with this much content with a cheap buy-to-play model and a totally skippable cash shop and no sub. It’s a low price with extraordinary value, and that makes it perfect for families, which is exactly why my kids play with me. The only other MMO that comes close is Guild Wars 1, and that’s not an accident."

A shout-out to OG Guild Wars as well!

Ice Drizzle

 

The shiny river looking area in the photo is the smoothest ice I've ever experienced.  It looks textured BUT IT IS NOT.  It is smooth AF.  (The texture is from the surface underneath.)

We had the biggest ice storm in our area since 1996.  I kept expecting a lot of rain, but really it just drizzled.  But it was drizzling super-cooled water which turned to ice when it hit the ground.  The result of this drizzly super-cooled water was 3D printed smooth AF ice that was impossible to walk upon without slipping. 

We had to break some of it up our driveway so a car could get back into the garage.  We were only able to clear a path for the tires of the left side of the car!  That took two of us two hours.  We had to use a pickaxe to break the ice.

Maeslantkering, the Netherlands

Maeslantkering

In Termination Shock, by Neal Stephenson, he writes about a structure in the Netherlands designed to prevent tidal surges from destroying inland property.  I thought it was something he'd made up until he mentioned a date when it was constructed that predates today (the book is set a bit into the future).

As I read his description, my brain kept saying, "He can't mean that, no, what?", but apparently, he did a rather good job of describing this thing in words, because when I looked it up, it matched what I had imagined!

Those big arms rotate out into the water, where they join up, and then sink, to create a barrier.

It's waves all the way down

This article called The Double Slit Experiment convinced me there is no wave/particle duality.

It's all waves and the particles are in fact an artifact of measurement - not as, in causing the wave function to collapse into a particle - but an actual artifact of the equipment used to measure the photon.

Also mentioned in the article is the "many worlds" explanation is nonsense.  For one thing - if universes are splitting right and left, then where does all the energy come from to power these universes?  It requires infinite energy.  And likewise - how fast do these universes come into being?  All of the universe as we know it splits?  How can you tell?  Observations are still limited by the speed-of-light.

It's waves.  Matter is waves.  So simple. 

Enchant, Vancouver BC, 2016

Small World Holiday 2019

I was lucky enough to sneak in a visit to Disneyland a couple of months before ... COVID.

Iced Tree

 

ArenaNet - more recognition!

 MMO Studio of the Year!  Two years running!

Congratulations everyone!

The not-a-bank bank

I went to one of Bank of America's newly remodeled branches.  It doesn't look like a bank inside.  There are no teller windows.

A woman with an iPad greeted me as I went in.  "This is a very confusing bank!", I said.

She said, "I've been in banking 15 years and once you get used to it you'll really like it."

I have no reason not to believe her.

But!  I would suggest that for people like me going into the not-a-bank for the first time a little map or sign suggesting where to start would be good.  As it was, the iPad woman was engaged with another customer when I first entered.  I had no idea what to do or where to get in line so I just stood there.  Which worked.  But what if there were two of us waiting?  Where do we wait and keep ourselves in order?

These are important UX questions.