Editing Work (section)

= Pyp2p =

Status: proof-of-concept
[[File:Work 1.png|thumb]]

I created Pyp2p as a way to do P2P networking in Python for my decentralised exchange PoC at a time when there were few other options (LibP2P did not exist.) Pyp2p has some novel features that aren’t available in similar systems. It supports ‘TCP hole punching’ which is a way to bypass NATs without setting up port forwarding rules. The most interesting part about Pyp2p is it’s address format.

Addresses of nodes in Pyp2p describe many pieces of information about the host network which allows Pyp2p to use multiple techniques to attempt to establish direct connectivity between two peers. It will attempt to first port forward with UPnP or NATPMP. If either side in a connection request have open ports then the connection can proceed. Otherwise Pyp2p has more tricks up its sleeve.

It profiles the type of NAT that peers are behind and then uses that to schedule TCP hole punching between the two peers. By combining different connection strategies it is possible to vastly increase the chances of achieving a direct connection between peers – even if both are behind strong NATs and firewalls.

<span id="what-i-learned"></span>
== What I learned ==

Working on this project I learned in-depth knowledge about networks and how messy P2P networking can be. I had to develop my own methodology for testing complex, time-sensitive protocols. I simulated networks with virtual interfaces and built fully functional simulations of peers behind routers using OpenWRT and VMware ESXi.

<span id="what-i-would-change"></span>
== What I would change ==

I came to this project with a heavy background in BSD sockets and C programming. It gave me a strong bias towards thinking in terms of threads, blocking and non-blocking sockets, and timeouts. In Python because it has the ‘global interpreter lock’ (GIL.) Every time you do a slow operation it stops the program until that operation has finished. In Python using I/O bound operations is a very, very bad idea and if I were to do this again I’d make everything async.

The other part of the program I’d change is how the hole punching works. You can say the GIL is concurrent but single-core. If you’re doing TCP hole-punching the whole process is ridiculously time-sensitive. Instead of trusting the GIL to multi-plex between a bunch of pending operations and starting the hole-punching code at the right time I would instead spawn a new Python process solely dedicated to hole-punching. The new process would occupy its own core and reduce the chances of competition tasks interrupting the scheduling.

It would also avoid slowing down the GIL of the main process by mistake and even allow for the priority of the new process to be increased which would further improve the accuracy of scheduling the hole punching. Finally, if it were a production project and not just a proof-of-concept, I’d completely re-write the code to avoid using god objects and stick to using small, loosely-coupled functions.