HelloZeroNet
/
ZeroNet
mirror of https://github.com/HelloZeroNet/ZeroNet.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'master' of github.com:HelloZeroNet/ZeroNet

This commit is contained in:
sirMackk 2015-02-22 11:40:44 -05:00
parent d1877085ac 531bf68ddd
commit de31a518dd
15 changed files with 274 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
README.md
View File

@ -1,22 +1,34 @@
# ZeroNet
Decentralized websites using Bitcoin crypto and BitTorrent network
Decentralized websites using Bitcoin crypto and the BitTorrent network
## Why?
- We believe in open, free and uncensored network and communication.
- No single point of failure: Site goes on until at least 1 peer serving it.
- No hosting costs: Site served by visitors.
- Impossible to shut down: It's nowhere because it's everywhere.
- Fast and works offline: You can access the site even if your internet is gone.
* We believe in open, free, and uncensored network and communication.
* No single point of failure: Site remains online so long as at least 1 peer
serving it.
* No hosting costs: Sites are served by visitors.
* Impossible to shut down: It's nowhere because it's everywhere.
* Fast and works offline: You can access the site even if your internet is
unavailable.
## How does it work?
- After starting `zeronet.py` you will be able to visit zeronet sites using http://127.0.0.1:43110/{zeronet_address} (eg. http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr).
- When you visit a new zeronet site, it's trying to find peers using BitTorrent network and download the site files (html, css, js...) from them.
- Each visited sites become also served by You.
- Every site containing a `site.json` which holds all other files sha512 hash and a sign generated using site's private key.
- If the site owner (who has the private key for the site address) modifies the site, then he/she signs the new `content.json` and publish it to the peers. After the peers have verified the `content.json` integrity (using the sign), they download the modified files and publish the new content to other peers.
* After starting `zeronet.py` you will be able to visit zeronet sites using
`http://127.0.0.1:43110/{zeronet_address}` (eg.
`http://127.0.0.1:43110/1EU1tbG9oC1A8jz2ouVwGZyQ5asrNsE4Vr`).
* When you visit a new zeronet site, it tries to find peers using the BitTorrent
network so it can download the site files (html, css, js...) from them.
* Each visited site becomes also served by you.
* Every site contains a `site.json` which holds all other files in a sha512 hash
and a signature generated using site's private key.
* If the site owner (who has the private key for the site address) modifies the
site, then he/she signs the new `content.json` and publishes it to the peers.
After the peers have verified the `content.json` integrity (using the
signature), they download the modified files and publish the new content to
other peers.
## Screenshot
@ -25,39 +37,45 @@ Decentralized websites using Bitcoin crypto and BitTorrent network
## How to join?
Windows:
- [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
- [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
- [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
- [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
- [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
- start `start.py`
Linux (Debian):
- `apt-get install python-pip`
- `pip install pyzmq` (if it drops a compile error then run `apt-get install python-dev` and try again)
- `pip install gevent`
- `pip install msgpack-python`
- start using `python zeronet.py`
### Windows
Linux (Without root access):
- `wget https://bootstrap.pypa.io/get-pip.py`
- `python get-pip.py --user pyzmq gevent msgpack-python`
- start using `python zeronet.py`
* [Install Python 2.7](https://www.python.org/ftp/python/2.7.9/python-2.7.9.msi)
* [Install Python ZeroMQ](http://zeronet.io/files/windows/pyzmq-14.4.1.win32-py2.7.exe)
* [Install Python Greenlet](http://zeronet.io/files/windows/greenlet-0.4.5.win32-py2.7.exe)
* [Install Python Gevent](http://zeronet.io/files/windows/gevent-1.0.1.win32-py2.7.exe)
* [Install Python MsgPack](http://zeronet.io/files/windows/msgpack-python-0.4.2.win32-py2.7.exe)
* Start `start.py`
### Linux
#### Debian
* `apt-get install python-dev python-pip`
* `pip install pyzmq gevent msgpack-python`
* Start with `python zeronet.py`
#### Without root access
* `wget https://bootstrap.pypa.io/get-pip.py`
* `python get-pip.py --user pyzmq gevent msgpack-python`
* Start with `python zeronet.py`
## Current limitations
- No torrent-like, file splitting big file support
- Just as anonymous as the bittorrent
- File transactions not compressed or encrypted yet
- No private sites
- You must have an open port to publish new changes
- Timeout problems on slow connections
* No torrent-like, file splitting for big file support
* No more anonymous than Bittorrent
* File transactions are not compressed or encrypted yet
* No private sites
* You must have an open port to publish new changes
* Timeout problems on slow connections
## How can I create a ZeroNet site?
Shut down zeronet if you are running it already
```
```bash
$ zeronet.py siteCreate
...
- Site private key: 23DKQpzxhbVBrAtvLEc2uvk7DZweh4qL3fn3jpM3LgHDczMK2TtYUq
@ -67,27 +85,35 @@ $ zeronet.py siteCreate
$ zeronet.py
...
```
Congratulations, you are done! Now anyone can access your site using http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
Congratulations, you're finished! Now anyone can access your site using
`http://localhost:43110/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2`
Next steps: [ZeroNet Developer Documentation](https://github.com/HelloZeroNet/ZeroNet/wiki/ZeroNet-Developer-Documentation)
## How can I modify a ZeroNet site?
- Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory. After you done:
```
* Modify files located in data/13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2 directory.
After you're finished:
```bash
$ zeronet.py siteSign 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
- Signing site: 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2...
Private key (input hidden):
```
- Enter your private key you got when created the site, then:
```
* Enter the private key you got when created the site, then:
```bash
$ zeronet.py sitePublish 13DNDkMUExRf9Xa9ogwPKqp7zyHFEqbhC2
...
Site:13DNDk..bhC2 Publishing to 3/10 peers...
Site:13DNDk..bhC2 Successfuly published to 3 peers
- Serving files....
```
- That's it! You successfuly signed and published your modifications.
* That's it! You've successfully signed and published your modifications.
## If you want to help keep this project alive
@ -97,5 +123,5 @@ Bitcoin: 1QDhxQ6PraUZa21ET5fYUCPgdrwBomnFgX
#### Thank you!
- More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
- Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet)
* More info, help, changelog, zeronet sites: http://www.reddit.com/r/zeronet/
* Come, chat with us: [#zeronet @ FreeNode](https://kiwiirc.com/client/irc.freenode.net/zeronet)

4
src/Config.py
View File

@ -3,7 +3,7 @@ import ConfigParser
class Config(object):
def __init__(self):
self.version = "0.2.1"
self.version = "0.2.3"
self.parser = self.createArguments()
argv = sys.argv[:] # Copy command line arguments
argv = self.parseConfig(argv) # Add arguments from config file
@ -19,7 +19,7 @@ class Config(object):
def createArguments(self):
# Platform specific
if sys.platform.startswith("win"):
upnpc = "tools\\upnpc\\upnpc-static.exe"
upnpc = "tools\\upnpc\\upnpc-shared.exe"
coffeescript = "type %s | tools\\coffee\\coffee.cmd"
else:
upnpc = None

20
src/File/FileServer.py
View File

@ -18,6 +18,7 @@ class FileServer:
else:
self.port_opened = None # Is file server opened on router
self.sites = SiteManager.list()
self.running = True
# Handle request to fileserver
@ -168,17 +169,26 @@ class FileServer:
if check_sites: # Open port, Update sites, Check files integrity
gevent.spawn(self.checkSites)
gevent.spawn(self.announceSites)
gevent.spawn(self.wakeupWatcher)
thread_announce_sites = gevent.spawn(self.announceSites)
thread_wakeup_watcher = gevent.spawn(self.wakeupWatcher)
while True:
while self.running:
try:
ret = {}
req = msgpack.unpackb(socket.recv())
self.handleRequest(req)
except Exception, err:
self.log.error(err)
self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
if self.running: self.socket.send(msgpack.packb({"error": "%s" % Debug.formatException(err)}, use_bin_type=True))
if config.debug: # Raise exception
import sys
sys.excepthook(*sys.exc_info())
sys.modules["src.main"].DebugHook.handleError()
thread_wakeup_watcher.kill(exception=Debug.Notify("Stopping FileServer"))
thread_announce_sites.kill(exception=Debug.Notify("Stopping FileServer"))
self.log.debug("Stopped.")
def stop(self):
self.running = False
self.socket.close()

3
src/Site/Site.py
View File

@ -194,9 +194,6 @@ class Site:
self.log.info("[OK] %s: %s" % (peer.key, result["ok"]))
else:
self.log.info("[ERROR] %s: %s" % (peer.key, result))
# Update content.json on peers

36
src/Ui/UiServer.py
View File

@ -11,20 +11,25 @@ from Debug import Debug
# Skip websocket handler if not necessary
class UiWSGIHandler(WSGIHandler):
def __init__(self, *args, **kwargs):
self.server = args[2]
super(UiWSGIHandler, self).__init__(*args, **kwargs)
self.ws_handler = WebSocketHandler(*args, **kwargs)
self.args = args
self.kwargs = kwargs
def run_application(self):
self.server.sockets[self.client_address] = self.socket
if "HTTP_UPGRADE" in self.environ: # Websocket request
self.ws_handler.__dict__ = self.__dict__ # Match class variables
self.ws_handler.run_application()
ws_handler = WebSocketHandler(*self.args, **self.kwargs)
ws_handler.__dict__ = self.__dict__ # Match class variables
ws_handler.run_application()
else: # Standard HTTP request
#print self.application.__class__.__name__
try:
return super(UiWSGIHandler, self).run_application()
except Exception, err:
logging.debug("UiWSGIHandler error: %s" % err)
del self.server.sockets[self.client_address]
class UiServer:
@ -89,4 +94,27 @@ class UiServer:
browser = webbrowser.get(config.open_browser)
browser.open("http://%s:%s" % (config.ui_ip, config.ui_port), new=2)
WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log).serve_forever()
self.server = WSGIServer((self.ip, self.port), handler, handler_class=UiWSGIHandler, log=self.log)
self.server.sockets = {}
self.server.serve_forever()
self.log.debug("Stopped.")
def stop(self):
# Close WS sockets
for client in self.server.clients.values():
client.ws.close()
# Close http sockets
sock_closed = 0
for sock in self.server.sockets.values():
try:
sock._sock.close()
sock.close()
sock_closed += 1
except Exception, err:
pass
self.log.debug("Socket closed: %s" % sock_closed)
self.server.socket.close()
self.server.stop()
time.sleep(1)

11
src/Ui/UiWebsocket.py
View File

@ -121,6 +121,8 @@ class UiWebsocket:
func = self.actionSiteSetLimit
elif cmd == "channelJoinAllsite" and "ADMIN" in permissions:
func = self.actionChannelJoinAllsite
elif cmd == "serverUpdate" and "ADMIN" in permissions:
func = self.actionServerUpdate
# Unknown command
else:
self.response(req["id"], "Unknown command: %s" % cmd)
@ -361,3 +363,12 @@ class UiWebsocket:
self.site.saveSettings()
self.response(to, "Site size limit changed to %sMB" % size_limit)
self.site.download()
def actionServerUpdate(self, to):
import sys
self.cmd("updating")
sys.modules["src.main"].update_after_shutdown = True
sys.modules["src.main"].file_server.stop()
sys.modules["src.main"].ui_server.stop()

19
src/Ui/media/Wrapper.coffee
View File

@ -46,6 +46,9 @@ class Wrapper
@sendInner message # Pass to inner frame
if message.params.address == window.address # Current page
@setSiteInfo message.params
else if cmd == "updating" # Close connection
@ws.ws.close()
@ws.onCloseWebsocket(null, 4000)
else
@sendInner message # Pass message to inner frame
@ -69,6 +72,10 @@ class Wrapper
@actionWrapperPrompt(message)
else if cmd == "wrapperSetViewport" # Set the viewport
@actionSetViewport(message)
else if cmd == "wrapperGetLocalStorage"
@actionGetLocalStorage(message)
else if cmd == "wrapperSetLocalStorage"
@actionSetLocalStorage(message)
else # Send to websocket
if message.id < 1000000
@ws.send(message) # Pass message to websocket
@ -126,6 +133,16 @@ class Wrapper
$('<meta name="viewport" id="viewport">').attr("content", @toHtmlSafe message.params).appendTo("head")
actionGetLocalStorage: (message) ->
data = localStorage.getItem "site.#{window.address}"
if data then data = JSON.parse(data)
@sendInner {"cmd": "response", "to": message.id, "result": data}
actionSetLocalStorage: (message) ->
back = localStorage.setItem "site.#{window.address}", JSON.stringify(message.params)
# EOF actions
@ -152,7 +169,7 @@ class Wrapper
@wrapperWsInited = false
setTimeout (=> # Wait a bit, maybe its page closing
@sendInner {"cmd": "wrapperClosedWebsocket"} # Send to inner frame
if e.code == 1000 # Server error please reload page
if e and e.code == 1000 and e.wasClean == false # Server error please reload page
@ws_error = @notifications.add("connection", "error", "UiServer Websocket error, please reload the page.")
else if not @ws_error
@ws_error = @notifications.add("connection", "error", "Connection with <b>UiServer Websocket</b> was lost. Reconnecting...")

38
src/Ui/media/all.js
View File

@ -120,17 +120,20 @@
}
};
ZeroWebsocket.prototype.onCloseWebsocket = function(e) {
ZeroWebsocket.prototype.onCloseWebsocket = function(e, reconnect) {
if (reconnect == null) {
reconnect = 10000;
}
this.log("Closed", e);
if (e.code === 1000) {
this.log("Server error, please reload the page");
if (e && e.code === 1000 && e.wasClean === false) {
this.log("Server error, please reload the page", e.wasClean);
} else {
setTimeout(((function(_this) {
return function() {
_this.log("Reconnecting...");
return _this.connect();
};
})(this)), 10000);
})(this)), reconnect);
}
if (this.onClose != null) {
return this.onClose(e);
@ -780,6 +783,9 @@ jQuery.extend( jQuery.easing,
if (message.params.address === window.address) {
return this.setSiteInfo(message.params);
}
} else if (cmd === "updating") {
this.ws.ws.close();
return this.ws.onCloseWebsocket(null, 4000);
} else {
return this.sendInner(message);
}
@ -807,6 +813,10 @@ jQuery.extend( jQuery.easing,
return this.actionWrapperPrompt(message);
} else if (cmd === "wrapperSetViewport") {
return this.actionSetViewport(message);
} else if (cmd === "wrapperGetLocalStorage") {
return this.actionGetLocalStorage(message);
} else if (cmd === "wrapperSetLocalStorage") {
return this.actionSetLocalStorage(message);
} else {
if (message.id < 1000000) {
return this.ws.send(message);
@ -891,6 +901,24 @@ jQuery.extend( jQuery.easing,
}
};
Wrapper.prototype.actionGetLocalStorage = function(message) {
var data;
data = localStorage.getItem("site." + window.address);
if (data) {
data = JSON.parse(data);
}
return this.sendInner({
"cmd": "response",
"to": message.id,
"result": data
});
};
Wrapper.prototype.actionSetLocalStorage = function(message) {
var back;
return back = localStorage.setItem("site." + window.address, JSON.stringify(message.params));
};
Wrapper.prototype.onOpenWebsocket = function(e) {
this.ws.cmd("channelJoin", {
"channel": "siteChanged"
@ -925,7 +953,7 @@ jQuery.extend( jQuery.easing,
_this.sendInner({
"cmd": "wrapperClosedWebsocket"
});
if (e.code === 1000) {
if (e && e.code === 1000 && e.wasClean === false) {
return _this.ws_error = _this.notifications.add("connection", "error", "UiServer Websocket error, please reload the page.");
} else if (!_this.ws_error) {
return _this.ws_error = _this.notifications.add("connection", "error", "Connection with <b>UiServer Websocket</b> was lost. Reconnecting...");

8
src/Ui/media/lib/ZeroWebsocket.coffee
View File

@ -66,15 +66,15 @@ class ZeroWebsocket
if @onError? then @onError(e)
onCloseWebsocket: (e) =>
onCloseWebsocket: (e, reconnect=10000) =>
@log "Closed", e
if e.code == 1000
@log "Server error, please reload the page"
if e and e.code == 1000 and e.wasClean == false
@log "Server error, please reload the page", e.wasClean
else # Connection error
setTimeout (=>
@log "Reconnecting..."
@connect()
), 10000
), reconnect
if @onClose? then @onClose(e)

16
src/User/User.py
View File

@ -25,9 +25,9 @@ class User:
self.log.debug("Saved")
# Get BIP32 address from site address
# Return: BIP32 auth address
def getAuthAddress(self, address):
# Get user site data
# Return: {"auth_address": "xxx", "auth_privatekey": "xxx"}
def getSiteData(self, address):
if not address in self.sites: # Genreate new BIP32 child key based on site address
s = time.time()
address_id = int(address.encode("hex"), 16) # Convert site address to int
@ -38,12 +38,17 @@ class User:
}
self.save()
self.log.debug("Added new site: %s in %.3fs" % (address, time.time()-s))
return self.sites[address]
return self.sites[address]["auth_address"]
# Get BIP32 address from site address
# Return: BIP32 auth address
def getAuthAddress(self, address):
return self.getSiteData(address)["auth_address"]
def getAuthPrivatekey(self, address):
return self.sites[address]["auth_privatekey"]
return self.getSiteData(address)["auth_privatekey"]
@ -51,3 +56,4 @@ class User:
def setData(self, data):
for key, val in data.items():
setattr(self, key, val)

2
src/User/UserManager.py
View File

@ -60,6 +60,6 @@ def getCurrent():
def reload():
import imp
global users, User
users.clear() # Remove all items
User = imp.load_source("User", "src/User/User.py").User # Reload source
users.clear() # Remove all items
load()

8
src/Worker/WorkerManager.py
View File

@ -1,5 +1,5 @@
from Worker import Worker
import gevent, time, logging
import gevent, time, logging, random
MAX_WORKERS = 10
@ -87,10 +87,12 @@ class WorkerManager:
def startWorkers(self, peers=None):
if len(self.workers) >= MAX_WORKERS and not peers: return False # Workers number already maxed
if not self.tasks: return False # No task for workers
for key, peer in self.site.peers.iteritems(): # One worker for every peer
peers = self.site.peers.values()
random.shuffle(peers)
for peer in peers: # One worker for every peer
if peers and peer not in peers: continue # If peers definied and peer not valid
worker = self.addWorker(peer)
if worker: self.log.debug("Added worker: %s, workers: %s/%s" % (key, len(self.workers), MAX_WORKERS))
if worker: self.log.debug("Added worker: %s, workers: %s/%s" % (peer.key, len(self.workers), MAX_WORKERS))
# Stop all worker

3
src/main.py
View File

@ -1,4 +1,5 @@
import os, sys
update_after_shutdown = False
sys.path.insert(0, os.path.dirname(__file__)) # Imports relative to main.py
# Create necessary files and dirs
@ -43,7 +44,6 @@ else:
import gevent
import time
logging.debug("Starting... %s" % config)
# Starts here when running zeronet.py
@ -56,6 +56,7 @@ def start():
# Start serving UiServer and PeerServer
def main():
global ui_server, file_server
from File import FileServer
from Ui import UiServer
logging.info("Creating UiServer....")

39
update.py Normal file
View File

@ -0,0 +1,39 @@
from gevent import monkey; monkey.patch_all()
import urllib, zipfile, os, ssl, httplib, socket
import cStringIO as StringIO
def update():
# Gevent https bug workaround (https://github.com/gevent/gevent/issues/477)
reload(socket)
reload(httplib)
reload(ssl)
print "Downloading.",
file = urllib.urlopen("https://github.com/HelloZeroNet/ZeroNet/archive/master.zip")
data = StringIO.StringIO()
while True:
buff = file.read(1024*16)
if not buff: break
data.write(buff)
print ".",
print "Extracting...",
zip = zipfile.ZipFile(data)
for inner_path in zip.namelist():
print ".",
dest_path = inner_path.replace("ZeroNet-master/", "")
if not dest_path: continue
dest_dir = os.path.dirname(dest_path)
if dest_dir and not os.path.isdir(dest_dir):
os.makedirs(dest_dir)
if dest_dir != dest_path.strip("/"):
data = zip.read(inner_path)
open(dest_path, 'wb').write(data)
print "Done."
if __name__ == "__main__":
update()

40
zeronet.py
View File

@ -1,13 +1,37 @@
#!/usr/bin/env python
def main():
try:
from src import main
main.start()
except Exception, err: # Prevent closing
import traceback
traceback.print_exc()
raw_input("-- Error happened, press enter to close --")
try:
from src import main
main.start()
if main.update_after_shutdown: # Updater
import update, sys, os, gc
# Update
update.update()
# Close log files
logger = sys.modules["src.main"].logging.getLogger()
for handler in logger.handlers[:]:
handler.flush()
handler.close()
logger.removeHandler(handler)
except Exception, err: # Prevent closing
import traceback
traceback.print_exc()
raw_input("-- Error happened, press enter to close --")
if main.update_after_shutdown: # Updater
# Restart
gc.collect() # Garbage collect
print "Restarting..."
args = sys.argv[:]
args.insert(0, sys.executable)
if sys.platform == 'win32':
args = ['"%s"' % arg for arg in args]
os.execv(sys.executable, args)
print "Bye."
if __name__ == '__main__':
main()
main()