Kenosis: BT tracker failover / distribution

[Gary Fung]

Hi all,

Came across Kenosis:
http://freshmeat.net/articles/view/1440/
http://kenosis.sourceforge.net/README_KENOSIS.txt

From TFA:

"Kenosis is a fully-distributed peer-to-peer RPC system built on top of XMLRPC. Nodes are automatically connected to each other via a Kademlia-style network and can route RPC requests efficiently to any online node. Kenosis does not rely on a central server; any Kenosis node can effectively join the network ("bootstrap") from any connected node."

"How does Kenosis-enabled BitTorrent work?

There are two cases to consider: legacy BitTorrent clients and Kenosis-enabled BitTorrent clients. In either case, imagine we have two Kenosis-enabled BitTorrent trackers being run on machines TA and TB (both on port 1234). Machines TA and TB automatically organize themselves into a Kademlia-style network via Kenosis. Let's further imagine a file being shared by the operator of TA, called FA. FA's .torrent file is constructed to talk to this tracker URL: http://foo.bt.kenosisp2p.org:1234/announce.

For Kenosis-enabled BitTorrent clients:
1. The client is asked to download the .torrent file for FA.
2. The client discovers the embedded URL http://foo.bt.kenosisp2p.org:1234/announce. Based on this URL, the client calculates a node address (NA) in the Kenosis network based on the sha1 hash of the key "foo".
3. The client does a Kenosis lookup for NA. Assume that FA is the closest node in the Kenosis network for node address NA. The client connects to the tracker on TA (port 1234) and downloads FA as normal.

For legacy BitTorrent clients:
1. The client is asked to download the .torrent file for FA.
2. The client does a DNS lookup on the URL http://foo.bt.kenosisp2p.org:1234/announce. The Kenosis-DNS bridge returns the IP address of FA.
3. The client proceeds to download FA as normal."


Their DNS bridge is what I'm doing with http://{hash}.bthub.com:{port}/announce tracker redirections. Kenosis is written in python, so it maybe worthwhile to investigate this as an extension, towards distributed BT trackers.
--
Cheers,
Gary



Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Elliott Mitchell]

Post by Gary Fung
"Kenosis is a fully-distributed peer-to-peer RPC system built on top of XMLRPC. Nodes are automatically connected to each other via a Kademlia-style network and can route RPC requests efficiently to any online node. Kenosis does not rely on a central server; any Kenosis node can effectively join the network ("bootstrap") from any connected node."

RPC and XML.

Implementations of both have been fraught with security holes. RPC is
pretty well guarenteed to be dropped at any decent border firewall, as
it it used for LAN-only protocols. Very poor choice for something that
will span the Internet. XML is big and complex. Why is something so big
and complex needed where robustness is needed?

Not to mention they still have some flavor of centralized tracker
otherwise things just don't work (okay, so I'm unsure what Kademlia-style
means). Notice earlier discussions on getting rid of the tracker, it just
doesn't work.

Post by Gary Fung
There are two cases to consider: legacy BitTorrent clients and Kenosis-enabled BitTorrent clients. In either case, imagine we have two Kenosis-enabled BitTorrent trackers being run on machines TA and TB (both on port 1234). Machines TA and TB automatically organize themselves into a Kademlia-style network via Kenosis. Let's further imagine a file being shared by the operator of TA, called FA. FA's .torrent file is constructed to talk to this tracker URL: http://foo.bt.kenosisp2p.org:1234/announce.

Sounds very much like the Multi-Tracker extension:
http://bittornado.com/docs/multitracker-spec.txt

It exists, it has been implemented. Haven't heard of complete failures
with it. Perhaps the Kenosis folks should consider begining on that?

Post by Gary Fung
Their DNS bridge is what I'm doing with http://{hash}.bthub.com:{port}/announce tracker redirections. Kenosis is written in python, so it maybe worthwhile to investigate this as an extension, towards distributed BT trackers.

And here (and above too) is that dirty little secret. You're using DNS as
your central server. Maybe not as much of one as the standard BT
protocol, but still very much a central server. If I'm an evil dude, I
can kill it by DoSing your nameservers, worse this kills ALL Kenosis
files. Sounds like a conventional P2P app, not BT.

So, what is new and different about this?
--
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/)
\ ( | ***@gremlin.m5p.com PGP 8881EF59 | ) /
\_ \ | _____ -O #include <stddisclaimer.h> O- _____ | / _/
\___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/





Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Gary Fung]

Post by Elliott Mitchell

Post by Gary Fung
"Kenosis is a fully-distributed peer-to-peer RPC system built on top of XMLRPC. Nodes are automatically connected to each other via a Kademlia-style network and can route RPC requests efficiently to any online node. Kenosis does not rely on a central server; any Kenosis node can effectively join the network ("bootstrap") from any connected node."

RPC and XML.
Implementations of both have been fraught with security holes. RPC is
pretty well guarenteed to be dropped at any decent border firewall, as
it it used for LAN-only protocols. Very poor choice for something that
will span the Internet. XML is big and complex. Why is something so big
and complex needed where robustness is needed?
Not to mention they still have some flavor of centralized tracker
otherwise things just don't work (okay, so I'm unsure what Kademlia-style
means). Notice earlier discussions on getting rid of the tracker, it just
doesn't work.

I agree XML is bloated relative to bencoding, but RPC security depends on the systems it hooks to. Changing encoding scheme also shouldn't be too difficult, consider this as a proof of concept.

I haven't read much about Kademlia either, but Kenosis describes a root node for discovery of peer nodes (trackers), and the location of root node is configurable. So there are central servers, but not a single one.

Post by Elliott Mitchell

Post by Gary Fung
There are two cases to consider: legacy BitTorrent clients and Kenosis-enabled BitTorrent clients. In either case, imagine we have two Kenosis-enabled BitTorrent trackers being run on machines TA and TB (both on port 1234). Machines TA and TB automatically organize themselves into a Kademlia-style network via Kenosis. Let's further imagine a file being shared by the operator of TA, called FA. FA's .torrent file is constructed to talk to this tracker URL: http://foo.bt.kenosisp2p.org:1234/announce.

http://bittornado.com/docs/multitracker-spec.txt
It exists, it has been implemented. Haven't heard of complete failures
with it. Perhaps the Kenosis folks should consider begining on that?

No this is different. multitracker pre-specifies trackers to connect to (I use it to add more bthub urls), Kenosis forms a P2P network of tracker nodes that all torrents can use. Node address is computed based on info_hash and a distance algorithm. Clients select the closest tracker node, if it goes down the next closest node is used so the swarm is not splitted.

Post by Elliott Mitchell

Post by Gary Fung
Their DNS bridge is what I'm doing with http://{hash}.bthub.com:{port}/announce tracker redirections. Kenosis is written in python, so it maybe worthwhile to investigate this as an extension, towards distributed BT trackers.

And here (and above too) is that dirty little secret. You're using DNS as
your central server. Maybe not as much of one as the standard BT
protocol, but still very much a central server. If I'm an evil dude, I
can kill it by DoSing your nameservers, worse this kills ALL Kenosis
files. Sounds like a conventional P2P app, not BT.
So, what is new and different about this?

See above. The DNS in kenosis serves as a bridge for clients that are not kenosis aware. It resolves the node address inside the DNS instead of in client. If you want to get rid of the DNS point of failure than compute node address in client.
--
Cheers,
Gary



Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Elliott Mitchell]

Post by Gary Fung
I haven't read much about Kademlia either, but Kenosis describes a root node for discovery of peer nodes (trackers), and the location of root node is configurable. So there are central servers, but not a single one.

That root node sounds an awfully lot like a tracker... ...and that
isn't even the central server I'd worry most about.

Post by Gary Fung

Post by Elliott Mitchell

Post by Gary Fung
There are two cases to consider: legacy BitTorrent clients and Kenosis-enabled BitTorrent clients. In either case, imagine we have two Kenosis-enabled BitTorrent trackers being run on machines TA and TB (both on port 1234). Machines TA and TB automatically organize themselves into a Kademlia-style network via Kenosis. Let's further imagine a file being shared by the operator of TA, called FA. FA's .torrent file is constructed to talk to this tracker URL: http://foo.bt.kenosisp2p.org:1234/announce.

http://bittornado.com/docs/multitracker-spec.txt
It exists, it has been implemented. Haven't heard of complete failures
with it. Perhaps the Kenosis folks should consider begining on that?

No this is different. multitracker pre-specifies trackers to connect to (I use it to add more bthub urls), Kenosis forms a P2P network of tracker nodes that all torrents can use. Node address is computed based on info_hash and a distance algorithm. Clients select the closest tracker node, if it goes down the next closest node is used so the swarm is not splitted.

Again, the root node sounds an awful lot like a tracker. True, subsequent
requests can go elsewhere, but that root node *must* be up in order for
clients to connect. What happens when I introduce an evil tracker? If
tracker nodes get created without human interaction, I can easily
introduce an evil tracker that kills everything.

Once you've brought human interaction into tracker creation, how is this
superior to Multi-Tracker? Certainly it is inferior because all initial
connections must be to that root hub. Where is the gain?
--
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/)
\ ( | ***@gremlin.m5p.com PGP 8881EF59 | ) /
\_ \ | _____ -O #include <stddisclaimer.h> O- _____ | / _/
\___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/





Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[a***@yahoo.com]

I wrote an elaborate piece some time ago about how the whole thing
just doesn't really help anything, and decentralization in software
is a myth, but my article shows where decentralization really exists
and how BT use has evolved to show that.

http://www.free-conversant.com/thom/main/2005/01/12

Thomas

Post by Elliott Mitchell

On Tue, 01 Feb 2005 18:59:10 -0800 (PST), Elliott Mitchell
I haven't read much about Kademlia either, but Kenosis describes

a root node for discovery of peer nodes (trackers), and the location
of root node is configurable. So there are central servers, but not a
single one.

Post by Elliott Mitchell
That root node sounds an awfully lot like a tracker... ...and that
isn't even the central server I'd worry most about.

Post by Elliott Mitchell

Post by Gary Fung
There are two cases to consider: legacy BitTorrent clients and

Kenosis-enabled BitTorrent clients. In either case, imagine we have
two Kenosis-enabled BitTorrent trackers being run on machines TA and
TB (both on port 1234). Machines TA and TB automatically organize
themselves into a Kademlia-style network via Kenosis. Let's further
imagine a file being shared by the operator of TA, called FA.
FA's .torrent file is constructed to talk to this tracker URL:
http://foo.bt.kenosisp2p.org:1234/announce.

Post by Elliott Mitchell

Post by Elliott Mitchell
http://bittornado.com/docs/multitracker-spec.txt
It exists, it has been implemented. Haven't heard of complete failures
with it. Perhaps the Kenosis folks should consider begining on that?

No this is different. multitracker pre-specifies trackers to

connect to (I use it to add more bthub urls), Kenosis forms a P2P
network of tracker nodes that all torrents can use. Node address is
computed based on info_hash and a distance algorithm. Clients select
the closest tracker node, if it goes down the next closest node is
used so the swarm is not splitted.

Post by Elliott Mitchell
Again, the root node sounds an awful lot like a tracker. True,

subsequent

Post by Elliott Mitchell
requests can go elsewhere, but that root node *must* be up in order for
clients to connect. What happens when I introduce an evil tracker? If
tracker nodes get created without human interaction, I can easily
introduce an evil tracker that kills everything.
Once you've brought human interaction into tracker creation, how is this
superior to Multi-Tracker? Certainly it is inferior because all initial
connections must be to that root hub. Where is the gain?
--
(\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)

___/)

Post by Elliott Mitchell
\_ \ | _____ -O #include <stddisclaimer.h> O- _____

| / _/

Post by Elliott Mitchell
\___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6

\_|_/___/






Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Bill Cox]

Hi.

It's been since June that Bram said anything about BT2. Is it
progressing?

BT needs an upgrade, and the ideas Bram mentioned for BT2 sound
promising. It's no good loading down the trackers. Merkle hash trees
are needed badly, as is tracker load distribution and redundancy.

Are there any code or protocol ideas we can start hacking together and
testing in the open-source community? We've got no official extension,
but at least two private companies are already using some BT2 ideas in
their proprietary protocols.

Thanks,
Bill





Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Blue Boar]

Post by Bill Cox
Hi.
It's been since June that Bram said anything about BT2. Is it
progressing?

Notice that there is a new Client beta as of about 2 weeks ago:
http://bittorrent.com/

BB



Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

[Olaf van der Spek]

Post by Bill Cox
Hi.
It's been since June that Bram said anything about BT2. Is it
progressing?
BT needs an upgrade, and the ideas Bram mentioned for BT2 sound
promising. It's no good loading down the trackers. Merkle hash trees
are needed badly, as is tracker load distribution and redundancy.
Are there any code or protocol ideas we can start hacking together and
testing in the open-source community? We've got no official extension,

I've got an experimental implementation of merkle trees in my client.

Post by Bill Cox
but at least two private companies are already using some BT2 ideas in
their proprietary protocols.
Thanks,
Bill
Yahoo! Groups Links

--
Olaf van der Spek
http://xccu.sf.net/



Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
BitTorrent-***@yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/