Post-Mortem: Hockey (Messenger)

Tuesday, September 17, 2024 by Nick

Image: Our competitor gets a nice pay day.

I used the brand name Hockey for three things: a messenger I started building in 2022, the C-Corp that’s the legal vehicle for my freelancing, and a social app I built in early 2023. This post is about the first one: the Hockey messenger.

Before leaving Pineapple in September 2022, I had a Zoom call with Eric Migicovsky. Former founder of the Pebble smartwatch, YC partner, and now CEO and founder of Beeper. Beeper is an app I didn’t believe was possible until I saw their landing page. A messenger app for your phone that combines all of your texts and inboxes into one app. They were compatible with iMessage, WhatsApp, Telegram, Slack, Signal, Discord, Facebook, Instagram, Twitter, and Linkedin

All your texts in one inbox. It sounded like magic! Eric got in touch with me because Beeper had a problem. They were building their company on top of the bridging technology Matrix, an open-source messaging protocol with a robust server implementation. The server technologies worked well, but the iOS reference app, “Element,” was slow, buggy, unreliable, and was mostly built on an outdated programming language. Beeper forked their messaging app off this genuinely horrible piece of tech and could not get it to the quality standards they were aiming for.

To fix this issue, they sought exceptional iOS developers to build a new app for them. And they found me. In the call that followed, I had a realization.

Matrix could be the future of messaging. This integration offers an unseen level of simplification in messaging. Beeper’s server technologies are all (mostly) open source. The closed-source client that Beeper is working on could be better. The client is Beeper’s core intellectual property (IP). If I were to build a better iOS app for Beeper, I would singlehandedly build a company’s core IP with incredible potential.

The last point is the most important one. I like working with clients on a cash compensation basis to limit my potential downside when the company fails to make money. I’m just building the tech; other people are running the company. My financial upside would be in the hands of a management team that I’m not a part of.

That’s why I do cash compensation: to limit my potential downside. Unfortunately, you can’t restrict that downside without limiting the potential upside. If Beeper had taken off and received a flat hourly fee for my work, I would have built between 10 and 60 percent of this company’s IP and moat while receiving no gain from it.

This fact drove me to start a new company and build Hockey, a matrix-based messaging client with modern iOS technologies. It was going to be the best messaging app on your phone—a better messenger that also replaces all other text apps from your home screen.

Product Market Fit

That product pitch sounded like a home run to me. I wanted this app badly, so I knew at least one potential customer was out there. In hindsight, I should have been more critical about being an outlier and considering my unique life circumstances. I was a European nomad living in Thailand and working for a company in Brooklyn. Daily, I talked to six different people on as many messaging apps. Like any social app, our target audience was young Americans, who mostly use iMessage and Instagram. The million-dollar question is: Do those users want to join these inboxes?

I talk to many of my friends on multiple apps at once. While this sounds like an imperfect setup and something to be solved, it doesn’t feel like a problem in the moment. I can send my girlfriend a date idea on TikTok, a meme on Instagram, and an urgent question on iMessage. There are distinct contexts for different communications.

The technical possibility that you could join all these messengers blew me away. I never stopped to ask if you should. I knew there was a chance that people wouldn’t be enthusiastic about the pitch. Still, I genuinely thought that the possibility was so slim that all user interviews felt more of a due diligence task to me rather than an actual vetting process.

I worked on this app with one co-founder and an advisor for a long time. What finally broke the camel’s back and shattered my rose-colored glasses was the painful and tedious process of watching our advisor onboard onto our app.

Meta doesn’t want you to ditch the Instagram app and use a third-party messenger. They know that messages are crucial to the Instagram feed’s retention. They show you more ads whenever you message someone. Connecting Instagram DMs and similar products to the hacky Matrix clients feels like pulling teeth. You need to turn on 2FA, log into the Matrix client, go back to Instagram, approve that the server-farm IP login was you, go back to Matrix, and slowly watch it sync all your messages—a three-minute process to connect one out of ten messengers. WhatsApp is even worse. You need a second device to show the connecting QR code, then open WhatsApp, go to settings, connect a “Desktop app,” and then scan the QR code that Matrix generated for you.

Getting the Matrix system running on our server was challenging, too. Their server “Synapse” doesn’t come as a simple docker-compose file or a Kubernetes manifest but is released as an Ansible playbook. That’s true for Synapse and all the messaging bridges that connect to it. I have never used Ansible before, and hopefully, I won’t use it again. It seems like a sub-par way to deploy software compared to containers. The playbook will SSH into your machine, run commands & tinker with system settings. It feels like I’m back in 2014, running apt-get install mysql-server.

Tech Wins

Many of the core technologies that make up Matrix feel suboptimal to me. I felt relieved when we finally decided to close this chapter and shut down the project. The prospect of my day job being a cat-and-mouse game with Instagram’s login API and fighting against Meta over their users didn’t feel like a great use of years of my life. Not to mention all the low-quality open-source technology I would have had to take over, maintain, and improve. Before Synapse was archived, the repo had 23,441 commits and 253,924 lines on the main branch.

With all that negativity out of the way, I want to celebrate some things that went right in this project. I made the most out of the hand I was dealt and am proud of the messenger I built.

For starters, I set up an automatic Fargate scaling system to spin up a container for every messaging bridge of every user, scaling our infrastructure dynamically with an arbitrary number of users.

On the iOS side, I removed all Matrix open-source code and decided to build a new SDK from scratch. I used Swift and Swift concurrency for a modern and performant library. I also fixed all the freeze frames and stutters that stemmed from the persistency layer of Element iOS’s data storage. If I remember correctly, they used Realm and didn’t do an excellent job passing data between threads. Plenty of blockers inside the Main thread were waiting for data tasks in a background thread—truly outstanding craftsmanship by the Element team. Instead, I built a new consistency layer from scratch using SQLite.

Unfortunately, this now surfaced another bottleneck. The sync endpoint took many seconds to compute a data diff on the server for the client to receive a new badge of messages. Every time you opened the app, you had to wait up to 1 minute to fetch the latest messages. That’s just bad design. I decided to fork the Synapse server and write a custom query to fetch only the newest message from every thread you’re a part of. Then, once you open a thread on iOS, all previous messages are paginated from the server on the fly.

The last component I fixed was the notification server “Sygnal.” Another crappy piece of software that didn’t have any support for grouping messages per thread and replacing notifications whenever a message gets edited or deleted. I retrofit that project to add those features.

With all these infrastructure improvements and a UI wholly rewritten in SwiftUI, Hockey provided a messaging experience that felt 1,000 times better and smoother than using Beeper. Instead of taking an average of 50 seconds to sync messages after connecting a new messenger, Hockey updated all your threads in less than 3 seconds. No matter if you have received five messages since you last opened the app or if 5,000 messages are downloading from WhatsApp. When opening the app, instead of waiting for the list of threads to load from Realm for up to 30 seconds, our app loaded all threads from SQLite in less than 1 second. You could interact with your conversations and send messages instantly.

After we had already abandoned Hockey, Beeper published a lot of documentation about their non-federated rewrite of Synapse. They called their server Hungry (a joke about being unfed). They seemingly reached similar conclusions about Matrix’s flaws as I did.

Closing Thoughts

Writing out these learnings for the first time has led me to the realizations above. Our product was clunky, and the solution was flawed. I was so focused on this app’s incredible technology breakthroughs that I didn’t spend enough time testing our hypotheses. I was blind to the possibility that the hypotheses could be false; they felt like facts. I could not subtract my views from this product and see it through the eyes of a non-technical stranger.

For the longest time, I blamed the VC down market for our inability to fund the product or my failure to deliver a great pitch. None of those things matter if you can build a great product that sells well. Successful products don’t require VC funding, and if you blame your inability to receive VC funding for your product’s failure, you need to flip the script in your head.