Tuesday, September 17, 2013
When GitHub is down, BitTorrent Sync saves the day!
BitTorrent Sync to the rescue! I have a directory full of bare repositories that I share with BitTorrent Sync. I just push there, then, by the time I'm back at work, the code has been copied to my computer here.
Monday, September 16, 2013
To the FairTax Proponents
Let me say again (because the rest of this post will probably make you think I don't support this legislation): I support the principles that FairTax seems to be built on. The principles seem to be comprehensibility, simplicity, fairness and a lack of loopholes.
But you, legislators trying to pass this bill, do not currently have my support. Mostly, because of problems with FairTax.org -- problems that have little to do with the content of the bill and everything to do with bad delivery and marketing.
1. Where are the Cons?
2. The FAQ is not helpful
I tend to agree with this article's estimation of the FAQ as a format. And after using the FairTax.org FAQ, I agree even more for two reasons:- That I have to click every single question before the answer shows is ridiculous. And I can't have more than one answer open at a time. Just give me the info! I can scroll past the information that isn't relevant.
- I read about how other governments (e.g. State of Florida) have implemented similar tax structures. About an hour later, I wanted to show my wife and I couldn't find the same question again. I found myself trying iterating through possible ways the question could be phrased. If all the text was on the page, I would have just searched the page for "Florida."
3. Link to the text
All of the text and videos on FairTax.org are nothing compared to the actual text of the legislation. You can say all you want, however you want, but it's not FairTax.org that's going to be put into law, it's H.R. 25. Please provide a link to the text of the bill. Then, rather than telling my what you think the bill says, quote the bill directly. A side benefit to quoting the actual text is that it might motivate the authors of the text to make it more intelligible to those who aren't politicians.4. Lead with 30%
5. Corporations are fictions?
Thursday, August 22, 2013
Angular AJAX Upload
| | |
Though the Internet would have you believe otherwise, uploading a file asynchronously from AngularJS isn't that hard. I don't want fancy colors or previews or progress bars or any of that. I want to upload a file from my AngularJS-backed webapp without reloading the page. Also, I don't care about old browsers. If you do, then this might not work for you.
After struggling with blueimp's library for way too long, I decided to just implement the part I needed.
Uploading a file using AJAX + AngularJS requires three things:
- AJAX
- AngularJS
- AJAX + AngularJS
1. AJAX
function upload(url, file) {
var formdata = new FormData(),
xhr = new XMLHttpRequest();
formdata.append('myfile', file);
xhr.onreadystatechange = function(r) {
if (4 === this.readyState) {
if (xhr.status == 200) {
// success
} else {
// failure
}
}
}
xhr.open("POST", url, true);
xhr.send(formdata);
}
The file will be posted to the server as the parameter named myfile.
2. AngularJS
app.directive('fileChange', function() {
return {
restrict: 'A',
link: function(scope, element, attrs) {
element.bind('change', function() {
scope.$apply(function() {
scope[attrs['fileChange']](element[0].files);
})
})
},
}
})
If you use the above directive like this:
<input type="file" file-change="runSomething">
when the user chooses a file to upload, runSomething will be called with a FileList. You can pass the first element in that list as the second arg to the upload function above.
3. AJAX + AngularJS
I can't provide a complete demo (because this blog isn't backed by a server I control). But this will probably get you really close:
<!DOCTYPE html>
<html lang="en">
<body ng-app="myapp" ng-controller="UploadCtrl">
<input type="file" file-change="upload">
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.0.7/angular.min.js"></script>
<script>
// the javascript
var app = angular.module('myapp', []);
//
// Reusable Uploader service.
//
app.factory('Uploader', function($q, $rootScope) {
this.upload = function(url, file) {
var deferred = $q.defer(),
formdata = new FormData(),
xhr = new XMLHttpRequest();
formdata.append('file', file);
xhr.onreadystatechange = function(r) {
if (4 === this.readyState) {
if (xhr.status == 200) {
$rootScope.$apply(function() {
deferred.resolve(xhr);
});
} else {
$rootScope.$apply(function() {
deferred.reject(xhr);
});
}
}
}
xhr.open("POST", url, true);
xhr.send(formdata);
return deferred.promise;
};
return this;
})
//
// fileChange directive because ng-change doesn't work for file inputs.
//
app.directive('fileChange', function() {
return {
restrict: 'A',
link: function(scope, element, attrs) {
element.bind('change', function() {
scope.$apply(function() {
scope[attrs['fileChange']](element[0].files);
})
})
},
}
})
//
// Example controller
//
app.controller('UploadCtrl', function($scope, $http, Uploader) {
$scope.upload = function(files) {
var r = Uploader.upload('/uploads', files[0]);
r.then(
function(r) {
// success
},
function(r) {
// failure
});
}
});
</script>
</body>
</html>
More
You can do more things like handle multiple files, monitor progress, preview images, etc... But if you don't need all that, and you are using modern browsers, this should do just fine.
Thursday, August 15, 2013
Practical event-driven programming with Python and Twisted
Introduction
A article from 2008 entitled Practical threaded programming with Python was posted to HN today. And I thought, "how would those examples look with Twisted?"
For a great explanation about how Twisted does concurrency, see krondo's Twisted Introduction. On to the code:
Hello World
The first example in the article demonstrates that threads have IDs. Since we're not using threads, the most equiavelent way to do the same thing with Twisted is to not use Twisted at all:
import datetime
def run(what):
now = datetime.datetime.now()
print '%s says Hello World at time: %s' % (what, now)
for i in range(2):
run(i)
Output:
0 says Hello World at time: 2013-08-15 13:45:17.164933
1 says Hello World at time: 2013-08-15 13:45:17.165442
Using queues
The next example shows first a serial approach and then a threaded approach to "grab a URL of a website, and print out the first 1024 bytes of the page." Here are the synchronous/serial and threaded versions.
I should note that I've modified them to get all the page (instead of the first 1024 bytes) and to print a hash of the content (so as not to clutter up this post). It's interesting that only apple.com and ibm.com return the same hash every time.
Synchronous version
import urllib2
import time
import hashlib
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
start = time.time()
#grabs urls of hosts and prints first 1024 bytes of page
for host in hosts:
url = urllib2.urlopen(host)
print hashlib.sha1(url.read()).hexdigest(), host
print "Elapsed Time: %s" % (time.time() - start)
Output:
2430771cc3723e965b64eda2d69dd22b697dd4a0 http://yahoo.com
790ace256c1b683a585226d286859f9f2910d9b0 http://google.com
63fbbe761817ebef066f9562e96209ca25a6f0b3 http://amazon.com
dd2f34c7c4f47b49272d7922e4f17f7c1cafd3aa http://ibm.com
562ffc06504dc0557386524b382372448d6e953a http://apple.com
Elapsed Time: 3.34798121452
Threaded version
#!/usr/bin/env python
import Queue
import threading
import urllib2
import time
import hashlib
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
queue = Queue.Queue()
class ThreadUrl(threading.Thread):
"""Threaded Url Grab"""
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
def run(self):
while True:
#grabs host from queue
host = self.queue.get()
#grabs urls of hosts and prints first 1024 bytes of page
url = urllib2.urlopen(host)
print hashlib.sha1(url.read()).hexdigest(), host
#signals to queue job is done
self.queue.task_done()
start = time.time()
def main():
#spawn a pool of threads, and pass them queue instance
for i in range(5):
t = ThreadUrl(queue)
t.setDaemon(True)
t.start()
#populate queue with data
for host in hosts:
queue.put(host)
#wait on the queue until everything has been processed
queue.join()
main()
print "Elapsed Time: %s" % (time.time() - start)
Output:
562ffc06504dc0557386524b382372448d6e953a http://apple.com
fb6fe32cb270f7929157bec5f29ee44f729949fd http://google.com
dd2f34c7c4f47b49272d7922e4f17f7c1cafd3aa http://ibm.com
3643a39f4dd641a3c08f8e5c409d0f5bc6407aed http://amazon.com
3072477b1680fc2650d9cb0674e5ef7972873bf6 http://yahoo.com
Elapsed Time: 1.23798894882
Twisted version
Here's one way to do the same thing with Twisted:
from twisted.internet import defer, task
from twisted.web.client import getPage
import time
import hashlib
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
start = time.time()
def printHash(content, host):
print hashlib.sha1(content).hexdigest(), host
def main(reactor, hosts):
dlist = []
for host in hosts:
d = getPage(host)
# when we have the content, call printHash with it
d.addCallback(printHash, host)
dlist.append(d)
# finish the process when the "queue" is done
return defer.gatherResults(dlist).addCallback(printElapsedTime)
def printElapsedTime(ignore):
print "Elapsed Time: %s" % (time.time() - start)
task.react(main, [hosts])
Output:
188eecd4da73515a9d1b3fde88d81ccc3a1e6028 http://google.com
562ffc06504dc0557386524b382372448d6e953a http://apple.com
dd2f34c7c4f47b49272d7922e4f17f7c1cafd3aa http://ibm.com
968fc83c1c7717575af03d43b236baf508134d0f http://yahoo.com
90c51ab729261bb72db922fb5ad22c0ae33c09da http://amazon.com
Elapsed Time: 1.36157393456
The run times of the threaded version and the Twisted version are comparable. Running them each multiple times, sometimes the threaded version is faster and sometimes the Twisted version is faster. They are both consistently faster than the synchronous version. Either way, this isn't a great benchmark and doesn't say much about how ansynchronous v. threaded will work in your particular case.
Working with multiple queues
The article's third bit of code shows how to use multiple queues to get the URL's body in one thread, then process it in another thread.
Threaded version
import Queue
import threading
import urllib2
import time
from BeautifulSoup import BeautifulSoup
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
queue = Queue.Queue()
out_queue = Queue.Queue()
class ThreadUrl(threading.Thread):
"""Threaded Url Grab"""
def __init__(self, queue, out_queue):
threading.Thread.__init__(self)
self.queue = queue
self.out_queue = out_queue
def run(self):
while True:
#grabs host from queue
host = self.queue.get()
#grabs urls of hosts and then grabs chunk of webpage
url = urllib2.urlopen(host)
chunk = url.read()
#place chunk into out queue
self.out_queue.put(chunk)
#signals to queue job is done
self.queue.task_done()
class DatamineThread(threading.Thread):
"""Threaded Url Grab"""
def __init__(self, out_queue):
threading.Thread.__init__(self)
self.out_queue = out_queue
def run(self):
while True:
#grabs host from queue
chunk = self.out_queue.get()
#parse the chunk
soup = BeautifulSoup(chunk)
print soup.findAll(['title'])
#signals to queue job is done
self.out_queue.task_done()
start = time.time()
def main():
#spawn a pool of threads, and pass them queue instance
for i in range(5):
t = ThreadUrl(queue, out_queue)
t.setDaemon(True)
t.start()
#populate queue with data
for host in hosts:
queue.put(host)
for i in range(5):
dt = DatamineThread(out_queue)
dt.setDaemon(True)
dt.start()
#wait on the queue until everything has been processed
queue.join()
out_queue.join()
main()
print "Elapsed Time: %s" % (time.time() - start)
Output:
[<title>Apple</title>]
[<title>Google</title>]
[<title>IBM - United States</title>]
[<title>Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more</title>]
[<title>Yahoo!</title>]
Elapsed Time: 1.65801095963
Twisted version
For this simple example, it makes sense to just do the processing right after receiving the body. That would look like this:
from twisted.internet import defer, task
from twisted.web.client import getPage
import time
from BeautifulSoup import BeautifulSoup
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
start = time.time()
def printTitle(content, host):
soup = BeautifulSoup(content)
print soup.findAll(['title'])
def main(reactor, hosts):
dlist = []
for host in hosts:
d = getPage(host)
# when we have the content, call printTitle with it
d.addCallback(printTitle, host)
dlist.append(d)
# finish the process when the "queue" is done
return defer.gatherResults(dlist).addCallback(printElapsedTime)
def printElapsedTime(ignore):
print "Elapsed Time: %s" % (time.time() - start)
task.react(main, [hosts])
Output:
[<title>Google</title>]
[<title>Apple</title>]
[<title>IBM - United States</title>]
[<title>Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more</title>]
[<title>Yahoo!</title>]
Elapsed Time: 1.80365180969
(As with the previous examples, neither the threaded nor the Twisted version are much different in speed.)
Hey!
"Hey! Those aren't the same!" I hear you say. You are right. They are not. The threaded version could extract the title in ThreadUrl.run instead of putting the content in queue for a DatamineThread.
I think the author was trying to show how you can make two threads work together on something... big? I haven't come up with a problem where it makes sense to write something in the Twisted version other than d.addCallback(printTitle, ...). If you have an idea post a comment, and I'll happily update this post (or make another post).
Conclusion
You can do things with threading. You can do things with Twisted. You should investigate Twisted (mostly for reasons not mentioned in this post). As noted above, krondo's Twisted Introduction is good, or there's some stuff I've written.
Also, if anyone can think of a better scenario for the two-kinds-of-thread-workers model, I'll update (or post again) with what a Twisted version might look like.
Wednesday, July 10, 2013
Angular injection
tl;dr is marked throughout by ∴
I don't like magical code. AngularJS is magical. I must fix this.
Dependency injection was one of AngularJS's first evil magicks I encountered. The idea that calling this function
function myFunction($scope, $http) {
...
}
$scope and $http runs contrary to all the JavaScript I've ever used. You can't do that! So I dug in to discover the magicks. And now it's not magic! It's great! It's rougly equivalent to import in Python or require in Ruby. Here's how it works:
Modules
AngularJS groups injectable things together into modules. The following code will:
- make a module named
woods - add a provider to the
woodsmodule namedEeyore, which has a constant value
var woods = angular.module('woods', []);
woods.value('Eeyore', 'sad')
Here's some of the source for the module function plus context (see the full source here — the comments are helpful):
// from setupModuleLoader()
function ensure(obj, name, factory) {
return obj[name] || (obj[name] = factory());
}
// ...
var modules = {};
return function module(name, requires, configFn) {
// ...
return ensure(modules, name, function() {
// ...
var moduleInstance = {
// ...
requires: requires,
name: name,
provider: invokeLater('$provide', 'provider'),
factory: invokeLater('$provide', 'factory'),
service: invokeLater('$provide', 'service'),
value: invokeLater('$provide', 'value'),
constant: invokeLater('$provide', 'constant', 'unshift'),
filter: invokeLater('$filterProvider', 'register'),
controller: invokeLater('$controllerProvider', 'register'),
directive: invokeLater('$compileProvider', 'directive'),
// ...
};
// ...
return moduleInstance;
// ...
});
};
- The
ensure(obj, name, factory)function makes sure thatobjhas an attribute namedname, creating it by callingfactoryif it doesn't. - The
module(name, requires, configFn)function adds amoduleInstancenamednameto the global-ishmodulesobject (by usingensure).
∴ angular.module(...) adds a module to some global-ish module registry.
Injectors
Injectors find providers from among the modules it knows about. By default, AngularJS creates an injector through the bootstrapping process. We can also make an injector with angular.injector() and use it to access providers within modules:
// Run this in a JavaScript console (on a page that has AngularJS)
// Make a woods module with an Eeyore provider
var woods = angular.module('woods', []);
woods.value('Eeyore', 'sad')
// Make an injector that knows about the 'woods' module.
var injector = angular.injector(['woods'])
// Get poor Eeyore out of the module
injector.get('Eeyore');
// -> "sad"
The creation of injectors and how they know where things are is somewhat recursive (and the code is a little hard to read). I will unravel that magic in another post as it was making this post too long. For now, just know that
∴ Injectors can find the providers you add to modules (e.g. through .value(...) or .factory(...)) and can find modules that were previously added to the global-ish module registry.
Invoke
Using an injector, we can invoke functions with dependency injection:
// Run this in a JavaScript console (on a page that has AngularJS)
// Make a woods module with an Eeyore provider
var woods = angular.module('woods', []);
woods.value('Eeyore', 'sad')
// Make an injector that knows about the 'woods' module.
var injector = angular.injector(['woods'])
// Imbue a function with sadness
function eatEmotion(Eeyore) {
return 'I am ' + Eeyore;
}
injector.invoke(eatEmotion);
// -> "I am sad"
But how does it KNOOooooowwwWWW??
How does AngularJS know the names of the arguments a function is expecting? How does it know that my weather function's arguments is named sunny?
function weather(sunny) {
...
}
That's an internal detail of weather, inaccessible from the outside, no? I've done introspection with Python, but this is JavaScript.
How AngularJS gets the argument names made me laugh out loud when I found it. It's a dirty (effective) trick found in the annontate function (full source):
var FN_ARGS = /^function\s*[^\(]*\(\s*([^\)]*)\)/m;
var FN_ARG_SPLIT = /,/;
var FN_ARG = /^\s*(_?)(\S+?)\1\s*$/;
var STRIP_COMMENTS = /((\/\/.*$)|(\/\*[\s\S]*?\*\/))/mg;
function annotate(fn) {
var $inject,
fnText,
argDecl,
last;
if (typeof fn == 'function') {
if (!($inject = fn.$inject)) {
$inject = [];
fnText = fn.toString().replace(STRIP_COMMENTS, '');
argDecl = fnText.match(FN_ARGS);
forEach(argDecl[1].split(FN_ARG_SPLIT), function(arg){
arg.replace(FN_ARG, function(all, underscore, name){
$inject.push(name);
});
});
fn.$inject = $inject;
}
} else if (isArray(fn)) {
last = fn.length - 1;
assertArgFn(fn[last], 'fn');
$inject = fn.slice(0, last);
} else {
assertArgFn(fn, 'fn', true);
}
return $inject;
}
∴ If you pass a function to annotate it will convert that function to a string and use regular expressions to get the names of the arguments.
I should note, however, that the practice of depending on argument names for injection is discouraged (because of how the names get munged during minification). It makes the code look cleaner, though. Maybe we should work on changing minification to handle this introspective kind of injection.
Which functions have it? Which don't?
When you're just starting with AngularJS, it's a little frustrating that some functions are magic (i.e. are called with injection) and some are seemingly inert. For instance, when writing a directive, link is not called with dependency injection, but controller is.
The provider methods are called with injection (factory, value, etc...). And directive controllers are called with injection. From the official docs:
DI is pervasive throughout Angular. It is typically used in controllers and factory methods.
∴ Sadly, the only way to know if a function is called with dependency injection is to... know. Read the docs or the source, and build up an ample supply of doing it wrong :)
Namespacing
Modules provided to an injector will stomp on each other's providers:
// Run this in a JavaScript console (on a page that has AngularJS)
function mineFor(Thing) {
return "I found " + Thing + "!";
}
// Make two modules that each define a Thing provider
var good_module = angular.module('good', []);
good_module.value('Thing', 'gold');
var bad_module = angular.module('bad', []);
bad_module.value('Thing', 'sour milk');
// Make an injector
var injector = angular.injector(['good', 'bad']);
injector.invoke(mineFor);
// -> "I found sour milk!"
I don't know if this is by design or if there are plans to address it. Be aware of it.
In summary
∴ Dependency injection in AngularJS is roughly equivalent to other languages' including and importing, but scoped to functions. Some of the magic is accomplished by exploiting function.toString() and regular expressions.
Read the official doc about Dependency Injection for some of the motivation behind its use.
Friday, May 24, 2013
Angular service or factory?
tl;dr is at the end
In various AngularJS tutorials and documentation, the authors choose to use service or factory but don't explain why you would use one or the other. Few mention that value and constant are also options.
Let's see why you would use one over the other. We should also understand how providers work:
provider
Here's the source for the provider method:
function provider(name, provider_) {
if (isFunction(provider_) || isArray(provider_)) {
provider_ = providerInjector.instantiate(provider_);
}
if (!provider_.$get) {
throw Error('Provider ' + name + ' must define $get factory method.');
}
return providerCache[name + providerSuffix] = provider_;
}
name is a string. provider_ can be one of three things:
- function
If a function is passed in, the function is called with dependency injection and should return an object with a
$getmethod. - array
An array will be treated like a function using Inline Annotation. It must also return an object with a
$getmethod. - object
If an object is passed in, it is simply expected to have a
$getmethod.
Whatever the second arg to provider is, you eventually end up with an object that has a $get method. Here's an example showing what happens:
// You can run this
// Create a module
var hippo = angular.module('hippo', []);
// Register an object provider
hippo.provider('awesome', {
$get: function() {
return 'awesome data';
}
});
// Get the injector (this happens behind the scenes in angular apps)
var injector = angular.injector(['hippo', 'ng']);
// Call a function with dependency injection
injector.invoke(function(awesome) {
console.log('awesome == ' + awesome);
});
Once you understand providers you will see that factory, service, value and constant are just convenience methods for making providers.
factory
Here's the source:
function factory(name, factoryFn) {
return provider(name, { $get: factoryFn });
}
So it lets you shorten the awesome provider creation code to this:
hippo.factory('awesome', function() {
return 'awesome data';
})
service
Here's the source:
function service(name, constructor) {
return factory(name, ['$injector', function($injector) {
return $injector.instantiate(constructor);
}]);
}
So it lets you make a factory that will instantiate a "class". For example:
var gandalf = angular.module('gandalf', []);
function Gandalf() {
this.color = 'grey';
}
Gandalf.prototype.comeBack = function() {
this.color = 'white';
}
gandalf.service('gandalfService', Gandalf);
var injector = angular.injector(['gandalf', 'ng']);
injector.invoke(function(gandalfService) {
console.log(gandalfService.color);
gandalfService.comeBack()
console.log(gandalfService.color);
});
The above code will instantiate Gandalf, but remember that everything that uses the service will get the same instance! (which is a good thing).
value
Here's the source:
function value(name, value) {
return factory(name, valueFn(value));
}
Using value would let you shorten the awesome provider to:
hippo.value('awesome', 'awesome data');
constant
Here's the source
function constant(name, value) {
providerCache[name] = value;
instanceCache[name] = value;
}
constant differs from value in that it's accessible during config. Here's how you use it:
var joe = angular.module('joe', []);
joe.constant('bobTheConstant', 'a value');
joe.value('samTheValue', 'a different value');
joe.config(function(bobTheConstant) {
console.log(bobTheConstant);
});
joe.config(function(samTheValue) {
console.log(samTheValue);
});
// This will fail with "Error: Unknown provider: samTheValue from joe"
var injector = angular.injector(['joe', 'ng']);
Read Module Loading & Dependencies in the Modules doc for more information on usage.
In summary
If you want your function to be called like a normal function, use factory. If you want your function to be instantiated with the new operator, use service. If you don't know the difference, use factory.
This is the (great) documentation for each function in the AngularJS source:
-
factoryA short hand for configuring services if only `$get` method is required.
-
serviceA short hand for registering service of given class.
-
valueA short hand for configuring services if the `$get` method is a constant.
-
constantA constant value, but unlike {@link AUTO.$provide#value value} it can be injected into configuration function (other modules) and it is not interceptable by {@link AUTO.$provide#decorator decorator}.
Wednesday, May 22, 2013
Angular and components
Something is going to come along and choose good names (without magic '@', '=', '&') with great documentation. Just today, I happened on this talk:
Which does a lot of AngularJS things.
I haven't explained thoroughly... sorry. Maybe when I have more time, I'll post more.
