Building a decentralized name system on top of IRC

The requirements

Over the past few years I’ve been slowly building a peer-to-peer networking library called ‘P2PD.’ The library is designed to make it easier to build peer-to-peer applications. But if I’m to build a library for software that’s supposed to be architecturally resilient – it makes sense for the library not to require centralization itself. My goal is that if I disappear the library should keep working without me having to manage things. The only way to make this possible is to design the system in such a way that it can utilize pre-existing public infrastructure. And it would be even better if that infrastructure were already serving multiple purposes to exclude the possibility of infrastructure disappearing. I know this is a little abstract but bare with me.

The problem

As an example: In P2PD there’s a need to be able to exchange meta-data signaling messages to peers. So given the requirements above – what protocols and infrastructure do you choose? Well, there are public servers that offer ‘pub-sub.’ MQTT is widely used and it’s used by enough devices that it’s unlikely to disappear. So if I have a list of stable, high-quality, MQTT servers, and build this into an address format to contact a node, then I don’t have to host these servers myself. I can disappear and it keeps working. Now, why am I saying all this? Because there was a piece missing: the DNS part.

DNS is the thing that makes the Internet user-friendly. But think about this: while there seems to be public infrastructure for all kinds of services on the Internet – nothing exists for something as simple as a permissioned key-value store. We can say that DNS is a paid, permissioned, key-value store, and that it’s good enough for the Internet today. But DNS has a few key problems. Firstly (1) there are no widespread standards to programmatically register domains and (2) DNS costs money to use (which I argue is not suitable for all uses.)

What alternatives are there to DNS? Well, there is Opennic – it’s actually great but again it’s a fork of the existing DNS system (and so has no standardized registration system.) There is the ‘Ethereum Name System’ – but it needs money to use (and we’ve learned the hard way that blockchains aren’t the right solution for everything.) There was no protocol for programmatic and free registration of names…

IRC as a DNS system

IRC is an ancient protocol used by multiple large networks. The protocol is popular for chat but flexible enough that all kinds of services have been built on top of it. A protocol based on IRC won’t easily die due to its long history as a chat protocol (there are IRC servers so old they’ve been online for decades.) Such infrastructure can be considered public, ubiquitous, open, and has withstood the test of time. Crucially: it has all the features needed to build a permissioned key-value store on.

The idea sounded promising to me in theory but before I went overboard I needed to verify if it would work. What I mean by that is would it be possible for a program to register a channel without the need for a human to verify their account? I knew that email registration was required on many popular IRC networks but I wasn’t sure if any were truly ‘open.’ So I manually tested about 20 different servers and eventually found one that allowed for account registration without email verification. This validated my theory that IRC could be used to build a name system. But how many of these servers were there?

To answer this question I needed to code a specialized scanner. The tool should test an IRC server by attempting to register an account and then register a channel. I rapidly wrote this code and used the server I already found to test it. Next I needed a list of IRC servers to scan. A big list. I ended up compiling every IRC server list from all major IRC clients by looking at their source code. The scanner could rapidly test servers as it was written to be asynchronous. I was able to find around 11 servers representing a prevalence of 1 server every 20 teste. Of these 11 servers – 5 were dual-stack and supported SSL.

I decided to reduce my list to only the dual-stack servers. The requirement for SSL was due to the fact IRC is a plaintext protocol and I needed to send a password over the connection. At least with SSL the passwords would be encrypted. Then I filtered the servers that supported both IPv4 and IPv6.

How it should work

For my design I wanted names to be stored across IRC servers as channels. It should be possible for the software to keep working if some of the servers go down for short periods. I wanted registration to succeed if registration worked on a minimum number of servers. And for name lookups to be able to get by on only needing to check a sub-set of servers (until getting enough results implied registration must have succeeded.) So while registration may require say: 5 / 7 channel registrations to succeed — lookup should only need 3 successes (2 is the failure number + 1 implies that registration must have succeeded for that name.)

Registering a name:
– Given N IRC servers
– At least M names must be registered (minimum)
– Where M = N – (N * 0.4)
– Measures are taken to make this atomic (don’t register any if M names can’t be registered)

Looking up a name:
– A name may be registered on any of the IRC servers.
– Given that there can be up to failures = (N * 0.4)
– Finding failures + 1 must imply a name was successfully registered.
– Names are grouped by an owner’s ECDSA public key.
– Lookups succeed when result no per pub key >= failures + 1.
– A lookup isn’t guaranteed and may fail if all servers are tested.

Password management with a single seed

The first decision to make was how to manage passwords. If I were to generate passwords for every account, the software would need to have a horrible amount of state… Probably with a database, with a schema, with just the right queries, with just the right everything, and all the code to manage that. To mitigate that I opted to use a cryptographic seed to deterministically generate passwords.


base64_encode(
    sha3_256(
        IRC_PREFIX + 
        ":pass:" + 
        irc_server_domain + 
        seed
    )
)

I chose SHA3 to avoid length-extension attacks; Base64 encoding was used as a way to avoid breaking plaintext IRC protocol messages. The prefix portion is just a unique number that helped me out with testing. Otherwise, I couldn’t register accounts without generating a new seed. Using the IRC servers domain results in a unique password per server. The inclusion of the “:pass:” portion is used to differentiate the construct from other similar hash-based derivations (later username, nick, and email.) These fields are the same but use a more restrictive encoding to be compatible with the IRC protocol. The encoding for them is base36 (0-9A-Z).

Channel names for ‘domains’

At first my design to convert names to channel names looked like this:


"#" +
base36_encode(
    hash160(
        dns_name
    )
)

The reason for using hash160 is there’s quite a small limit for the length of a channel name. I have found it can be reliably assumed to be at least 32 bytes. Hash160 returns 20 bytes that need to be encoded in base32 to work as a channel name. That increases the size by reducing the symbols that can be used in a byte. This encoding scheme should allow for the digest to be stored in the channel name. But there is one more problem. Suppose you want to register a name and an attacker is watching the channel list. The moment they see your channel name they can quickly register the name on the other IRC servers potentially hijacking the name.

The fix I came up with is inspired by so-called ‘decentralized exchanges’. In order to prevent race conditions in name registration: names will be uniquely determined by a time-lock encrypted nonce. The registering party can “save-up” the time-locks in advance so that every channel name can be registered at once. The channel names will further be derived per server so that attackers need to decrypt the initial time-lock to determine the masked name for registration on other servers: by which time the channel names will have already been registered by the names rightful owner. The construct is as follows.


def get_chan_name(name, tld, irc_server, optional_pw=""):
    msg = "{name} {tld} {optional_pw} {irc_server_domain}"
    time_lock = argon2(
        msg,
        salt="some custom salt",
        time_cost=2, # Iterations
        memory_cost=1024 * 2, # KB needed.
        parallelism=1 # Threads needed.
    )   
    
    "#" + 
    base36_encode(
        hash160(
            sha3_256(
               msg + time_lock
            )
        )
     )

Argon2 is a nice hash algorithm as you can adjust memory usage, time cost, and threading. The final output of Argon2 becomes the time-lock. Here sha3_256 is used again but this time inside hash160. The reason for this is stacking hash functions can help reduce the chances of finding collisions (you would need to find a collision in both functions.) Note the inclusion of the specific IRC server hosting the ‘name’ and the introduction of a TLD. The user can be encouraged to use anything for the TLD so this becomes a quasi-password.

If an attacker wanted to generate rainbow tables they would need the TLD used and Argon2 would massively increase the cost of generating such a table. The introduction of an optional password field allows for names to be masked with a password. Attackers would then need to know the password in addition to the TLD and name.

Channel topics to store values

IRC channel names translate the names in this system while topics store their values. Since this is a ‘threshold’ design where a sub-set of servers may fail the topic format needs to account for that. I initially chose a scheme that contained a version, ECDSA public key, record signature, and record portion. The good thing about the topic field is the characters it can support are quite extensive. In my tests every server supported unicode topics. However, I decided to use a more restrictive encoding scheme (using only the full range of characters on a standard keyboard) to ensure that binary data would be safely encoded across servers.


ecdsa_priv = sha3_256("{name}{tld}{pw}{seed}")
ecdsa_keys = ECDSA.generate(ecdsa_priv)
topic =  "p2pd.net/irc" + " " +
base92_encode(ecdsa_keys.pub) + " " +
base92_encode(ecdsa_keys.sign(record)) + " " +
base92_encode(record)

Names are owned by public keys. Their ECDSA private key is built from a hash. Since it’s possible a resulting key will be invalid. The private key is hashed until the result is a valid key. Shortly after this I remembered an old trick that allowed the ECDSA public key to be recovered from a signature if you have the message. The recovery returns multiple keys that need to be saved. The original public key can be determined by creating a list of public keys for each topic component and choosing the one that passes the validity threshold (of course: keys still need to be able to verify signatures.) Thus, I was able to shorten the topic portion to three fields.


def encrypt(msg, otp):
    # Make otp long enough.
    while len(otp) < len(msg):
        otp += sha3_256(otp).digest()

    # Xor MSG with OTP.
    buf = ""
    for i in range(0, len(msg)):
        buf += msg[i] ^ otp[i]

    return buf
    
record = encrypt(record, time_lock)

Above is a simple algorithm for a one-time pad that uses hashing to do key-stretching. It is used as a way to encrypt the record portion of the topic and signature portions without increasing the message size. The fields are encrypted with encrypt(field, sha3_256(“{field_name}:” + time_lock)) so that each field uses a different pad. The reason the signature portion is encrypted is that names across servers could be correlated by observing which names share the same public key (like mentioned above.) The security guarantee for this is at least tied to the time-lock provided by Argon2 across the name meta-data.

Handling expiry

When nicknames and channels are registered on an IRC server they usually expire if they’re not used. The duration that nicknames and channels stay active is generally between 12 – 60 days. This means that being able to monitor how close accounts and channels are to expiry is important. I’ve created a basic function that automatically handles expiry and refreshes everything. The refresh code also attempts to register names on servers that had previously failed (where servers may have temporarily been down.)

In IRC there’s a few interesting mechanisms that might help prevent channels from being lost. (1) IRC supports ‘successors’ that gain control over a channel if the owner’s account expires. (2) Another option to help prevent channel expiry would be to utilize bots. This would be very easy to make low-trust. Naturally, names and accounts are automatically ‘refreshed’ when they’re used.

Mitigating disruption

I didn’t want to create something that may negatively impact IRC service. So my software takes a number of measures:

  1. The channels registered are set to private to avoid flooding the channel listing.
  2. Channel names are uniquely determined by an algorithm that requires CPU work to be done from clients to map names to values. This sets a limit on channel registrations and is required by the protocol for security.
  3. Channels will expire naturally due to chanserv optimizations and the software doesn’t aggressively attempt to refresh channels (refreshing nicknames and channels has to be done manually by the user.)
  4. The design means that lookups for names can be load-balanced across servers.

My software currently has few users so speculation about possible disruption remains theoretical. But if it should become a problem in the future I’d be happy to work with IRC operators to minimize abuse.

Usage for the software

That’s really all I have to say for now. I worked hard to build a prototype over the last month and have a basic library that can be used now. Details of the software can be seen at https://p2pd.readthedocs.io/en/latest/built/irc_kvs.html I’m also looking for a new role so if anyone is interested in working with someone who is reasonably skilled and can think outside the box hit me up.

My resume is here: https://github.com/robertsdotpm/resume/blob/main/resume.pdf

Merry Haxmas!

The web is boiling

A cute little froggy sits inside a pot and doesn’t notice as the water boils it. This is the modern web. Owned by billionaires and conglomerates. They will take you in promising to stay ‘open’, ‘free’, and ‘democratic.’ Over time people’s trust in these entities increases and many come to rely on them… only for them to screw people the moment they step out of line.

What you do doesn’t matter. It could be an automated system that gets abused… or perhaps something more insidious like deciding certain political beliefs constitute ‘trolling.’ The provider gets to say what is ‘acceptable.’ And as long as a person shares the same values everything goes okay. But occasionally… Something happens. Then you realize that everything you ‘had’ was just an illusion.

You never had a choice. You never had control over your ‘account’, your ‘data’, or ‘services.’ You were being kept inside a pen. The funny thing is. If the platform ever does anything bad (like loosing all your data which happens all the time) they’ll get a fine and go on their merry way. But you have zero recourse for being screwed over yourself.

Welcome to the ‘free’ web in 2023. Destination: you.

I tried the future of food and it wasn’t soylent

Soylent is a powdered food that promises to keep its adherents alive and give them back all the time they wasted on cooking. It comes in many different flavors and over the years its formulation has evolved to mitigate some of the side effects of consuming such a calorie-rich liquid (namely bloating, gas, and digestive issues.) Among certain circles Soylent has a cult following. I myself have tried some of its early formulations and saw the potential early on. But to me there is no getting around the obvious drawback of Soylent: which is it takes away the joy of eating food.

No matter how many flavors it comes in. One eventually gets tired of drinking it. These meal replacement companies focus on powders, drinks, and bars because they’re easy to scale. But actually using them as a viable way to save time cooking is ah… a little deluded. Reducing life to a chemical composition is a robotic way to say ‘fuck you’ to your humanity. It turns out that living on a chemical sludge is no where near as fun as it sounds. But I think there’s a much better alternative to what Soylent is trying to do and it’s called ready-meals. Mind you, I’m not talking about the ready-meals you get in the freezer isle at the local super market, no. I’m talking about ready-meals that are made by hand with fresh ingredients and haven’t undergone processing for long-term storage (e.g. high sodium, vacuum heating, all that stuff that destroys flavor.)

There are now many companies that offer prepared meals that you can refrigerate for immediate consumption or freeze for long-term storage. And the meals are delicious. Chef-prepared, restaurant-quality dishes that are cheaper and healthier than take-away. It’s still more expensive than buying groceries and cooking. But in exchange you get professionally prepared meals that only need 3 minutes in the microwave to eat. No defrosting. No high sodium. No other bullshit. Fresh ingredients. Does that sound better than drinking a chemical sludge? I think so.

During COVID there were a few enterprising people who started businesses cooking meals in their kitchens and selling it. That’s another approach to the problem. There have been some companies that focus on ‘peer-to-peer’ marketplaces for home cooked meals (I definitely would be a customer.) Though I’m not aware of any popular ones. I’d say if anyone is interested in trying these fresh ready-meals: do some Google research as you will need to find the companies that ship to your state. The meals arrive in cooled boxes, delivered in a refrigerated truck. Again, these meals are specifically not frozen so unless you put them in the freezer they will keep for about a week in the fridge.

Free public blockchains

For the people interested in blockchains the proposal is something like this: a shared database recording transferable credits is maintained by the introduction of rewards tied to the computational cost of producing them. In so doing, the cost of degrading trust within the system is tied to the cost of ever increasing computational complexity.

So with each successive reward prior transaction activity becomes more and more certain. The design here addresses the problem of allocating or minting credits within the system by making them the outcome of securing ledger activity. Perhaps implicit in this assumption is that a reward is needed for network participants to provide services. It’s a fair assumption, too. Why would people provide resources for free? But since this assumption also dictates how people use the transaction ledger its worth examining.

So I ask a very simple question: Are there any consensus-like systems out there that specifically don’t require fees to use? As it happens – the answer is yes. The closet system I’ve found so far is a project called OpenNIC. OpenNIC are an alternative authoritative registrar for domain names. So why is this important? Well, OpenNIC is doing something very similar to a blockchain system but they’re specifically NOT tying it to money, fees, or rewards. In OpenNIC there are different domain name servers to manage TLDs but lets simplify this and say that there’s still a level of consensus over the state of records in the system. OpenNIC are providing this service because:

(1) It’s literally a critically important service to the Internet
(2) Running an alternative that’s user-managed allows for more creativity
(3) It’s more open than the current system and costs nothing for users

This is provided for free by volunteers because they believe in the idea. Because they see value in it. Because it does provide value. So my question is: wouldn’t it also be possible to make a blockchain system that supports smart contracts that’s 100% free to use (no transaction fees required) using a similar model to OpenNIC. It would be much more useful for many apps because as stated already — even just DNS is a critical need for the Internet. A smart contract system that is free to use would be a better fit for the Internet than what exists.

Get Dell XPS 17 9730 setup with linux (and fix broken audio ‘dummy output’ and touchpad freezes)

I just bought the new Dell XPS 17 9730 laptop (and I really love it) but I had trouble (1) getting Linux installed on it (2) getting the audio working and (3) stopping the touchpad from freezing.

If you have these issues:

  1. Ensure ‘secure boot’ is disabled in the BIOS. For your own sanity you’ll probably also want to turn off ‘fast boot’ and set the delay to 5 seconds. Pressing F12 is how you get to the BIOS.
  2. Download OpenSUSE. On Windows you can use Rufus to make a bootable USB or Etcher for other OS’. The XPS only has USB C ports and if you try to boot from a smart card it will hang. You may need a dongle that gives you regular USB.
  3. Why OpenSUSE? Well, the simple reason is after wasting hours of my life OpenSUSE was the only OS where I got the audio working. So yeah, that’s what you get. Dell XPS 17 (the most recent version) has very new hardware and this has issues in current Linux distros.

Here’s the magic command on OpenSUSE to get your audio working:

sudo zypper install sof-firmware

After that’s installed reboot your system and your audio devices will show up. The last step is to click the speaker icon in the bottom right and select the output ‘Speaker (sof-soundwire Speaker)’ to use the speakers on your laptop. I haven’t checked bluetooth audio or the headphone port but I was able to get my airpods working on Ubuntu earlier so I don’t think it would be a big deal to use on OpenSUSE.

Overall this is a good laptop. The screen brightness matches a Mac Book Pro. The keyboard feels nice to type on and the hardware is very up to date. It is expensive but you always have to pay for quality.

Touchpad freezes or ‘sticking’

On OpenSUSE I’ve found that using the touchpad will cause the cursor to periodically get stuck. If you disable ‘hardware acceleration’ in your web browser the touchpad will function properly.

Edit: here’s how to get Airpods pro 2 working on OpenSUSE:

sudo zypper install blueman nano

sudo zypper install bluez-auto-enable-devices

sudo nano /etc/bluetooth/main.conf

[General]
DiscoverableTimeout = 0
ControllerMode = bredr
Experimental = true

[Policy]
AutoEnable=true

sudo nano /etc/pulse/default.pa
find the line that says:

load-module module-card-restore

and change it to:

load-module module-card-restore load-module module-card-restore

To enable your Airpod pros microphone:

open blueman, right click airpods, and change the audio profile to hands free

When you restart your audio and mic may be disabled again.
Running this script will fix that.

!/bin/bash
sudo su -
pkill blueman-manage || 1
systemctl restart bluetooth || 1
logoff

blueman-manager


Curls security announcement

Recently the curl project posted an announcement on Github that there was a severe vulnerability in curl and they would have a fix out in a week. Many people seemed to think that this notice was appreciated and gave people enough time to learn about the issue and know they would have to update in the future. But I’d argue the whole approach curl has taken is ultimately undesirable and put users at risk for no reason.

What is curl?

Curl is both a command-line tool and a library. It’s used widely for working with web requests – though I understand it supports other protocols. Curl is incredibly useful. It’s available on virtually every operating system. Many important projects use its library for working with common Internet protocols. Curl is a ubiquitous tool for today’s technologists.

Why was the notice bad?

When a security issue is found in software it’s common practice to keep knowledge about it internal until it’s fixed. This is done for several reasons:

  1. It avoids giving attackers knowledge of a vulnerability which they may use to exploit systems before a fix is available.
  2. It avoids creating unnecessary panic before a fix is out.
  3. It affords time to get patches out before a release is made.

What curl did was say there was a critical vulnerability while giving stakeholders nothing to protect themselves from it. While they didn’t publish details for a working exploit: for all we know they’ve motivated hoards of attackers to find an exploit and use it. There may also have been parties that already knew about the exploit for years and now they’re motivated to use it.

B-but it gave me time to prepare

People have said that the curl notice was appreciated because it gave affected parties time to prepare for the fix. But I don’t buy that. The reason is the timeframe – 1 week is not long enough to do anything. By saying there’ll be a patch available in a week they’ve forced a mandate on everyone that says ‘everyone currently using curl needs to update within a week or potentially get fuqt (not that severe – guessing it still requires interaction.)’

Unfortunately, it’s going to take time to get the fix out to users. Every distro will have to update packages. Entire tools will need to be written to scan for vulnerable versions of curl. Popular image hosting websites will need to update scores of images (that’s A LOT of builds.) They haven’t given people time to prepare. In fact, by publishing their little notice they’ve robbed them of time since the moment a fix is published in curl it will be possible to reverse-engineer an exploit and most systems still won’t be patched.

So in essence: with this notice they’ve decided on a deadline for all other vendors. Lest they leave their users vulnerable… and the dead line may not be sufficient. I would certainly consider that approach quite amateurish and negligent. A good case-study on how not to handle a security issue.

So what should curl have done?

The correct approach is simple. You privately take as long as you need to and write the fix FIRST. You privately message trusted parties at various mailing lists that include curl in their distros to coordinate the availability of a patched package on a certain date. You could also work with version control companies and image hosts to let them update tools to scan for vulnerable versions of curl and you wouldn’t even have to share the exploit specifics in such a scenario. THEN you write the notice… WITH a patch available.

People have pointed out the above already. But apparently having a different opinion to curls circle-jerk of spastics is considered trolling to them. Ironically, their very own security program asks that researchers report security problems privately. What a fucking joke. I think it’s safe to say you can ignore that.

Overwatch: hidden aim jutsu

In the sport of archery there are physical techniques to ensure consistent and accurate aim. In first person shooter games we don’t often think of it as a physical process – but it is. You’re familiar with the process of sitting down to use a mouse – but there are other physical practices that can be used to unlock god-tier aim. Here are my key technique to having consistent aim in Overwatch and similar FPS games.

Keep your head level and straight.

Your inner ears are the centre of balance for the entire body. When you keep your head level and straight something interesting happens when you aim. Firstly, because the screen is no longer shaking relative to your vision – targets become much easier to see.

You may be familiar with the phenomenon of your eyes ‘glazing over’ as targets on the screen become more chaotic and you start tracking them only superficially (missing shots in the process.) This virtually doesn’t happen when you keep your head straight.

Secondly, because targets are easier to track, it ends up becoming a matter of mechanical skill whether you hit targets. You can start to develop consistent, reproducible aim, even when tired.

Relax your body when you shoot.

During matches there is a lot going on which produces stress. When you’re stressed you will miss more shots. To practice this technique you should focus on slightly relaxing the muscles in your body as you shoot. This technique will help manage stress in-game and it will also help prevent shaky aim.

Directly look at the target to shoot.

I know this seems obvious but it’s not always easy to do. When targets move erratically your eyes become taxed from having to constantly focus. Sooner or later you may start relying more on peripheral vision and overall prediction to hit shots than direct focus. That is when you’ll miss.

The challenge is finding a way to maintain precise focus on targets you shoot at all times. Will you need to blink more? Will you need to occasionally look away to recharge? You will need to track the target precisely with your eyes at all times otherwise hitting them is just luck.

Don’t move the mouse with your clicks.

Imagine you’ve done everything right. You’ve lined up the perfect shot: the cross-hair is on the target. So you go to shoot and still you miss. Why is that? Well, if you’re anything like me just the act of going to click is enough to move the mouse ever so slightly. So that by the time you end up clicking your cross-hair is no longer on the target.

What you need to do is shoot in such a way that clicking doesn’t move the mouse at all. The best hero to learn this technique with is Hanzo on a custom game because you’re not going to get headshots consistently if you can’t master clicking without your mouse moving around.

If you master this last technique it will take your aim to a level you never thought possible. You’ll be able to do feats that seem super natural.

To recap:

  1. Keep your head level and straight.
  2. Relax your body when you shoot.
  3. Ensure you precisely track targets with your eyes.
  4. Don’t move the mouse with your clicks.