Sunday, July 13, 2014

DjangoRESTFramework, AngularJS and working with URLs

I like to put constants, URL's, etc into a django view that returns JSON for my angular apps that use django as a backend. You can write an angular service that calls this view, fetches and initialises this data for other angular services to consume by creating one service for constants, one for URL-mappings, etc. Or you can use a jsonify template tag like the one here and directly embed it in a script tag into your index template. I prefer the latter - I don't think it's worth the extra request just to keep your angular templates and your django templates separate.
<script type="text/javascript">
    var CONF = {
        urls: {{ urls | jsonify }},
        constants: {{ constants | jsonify }}
    };
</script>

With DjangoRESTFramework
Suppose you have endpoints, like the ones below, registered using a router in DRF,
api_router = DefaultRouter()
api_router.register("products", ProductViewSet)
api_router.register("product_notes", ProductNotesViewSet)
you should be able to enumerate all URLs for a given api router and build mappings for them using the following snippet.
def get_urls_by_router(api_router, urlconf=None):
    urls = {}
    list_name = api_router.routes[0].name
    for prefix, _viewset, basename in api_router.registry:
        urls[prefix] = reverse(list_name.format(basename=basename), urlconf=urlconf)
    return urls
print get_urls_by_router(api_router)
# {"products": "/api/v1/product/products", "product_notes": "/api/v1/product/product_notes"}
And in angular, I use a service, like the one below, to refer to these URLs by name instead of hardcoding them all over the place.
module.factory("URL", function(conf){
    var urls = conf.urls;
    return function(name){
        if(urls.hasOwnProperty(name))
            return urls[name];
        else
            throw new Error("Unknown URL identifier: " + name);
    };
});
where conf.urls is a mapping of URL names to URL patterns.

Special note about Multi-tenant systems
Suppose you are using the Sites framework to run multiple sites on a single django instance, you are probably using separate urlconfs for each site (so that Site1's URLs aren't visible from Site2). In cases like this, the URL you are trying to reverse might not be part of the request urlconf (available in request.urlconf). The urlconf argument to get_urls_by_router() is for cases like this where you have to override the request urlconf.

Cross-Domain requests
Sometimes, you might choose to serve your REST endpoints and the rest of your app from different domains (for eg., your actual app is at app.com while API is at api.app.com). For this, I used Django CORS headers to CORS enable my views. I use something like this to refer to API URLs in my code if the API is hosted on a different domain.
module.factory("APIURL", function(URL, constants){
    var api_base_url = constants.api_base_url;
    return function(name){
        return api_base_url + URL(name);
    };
});
where constants.api_base_url can be something like 'http://api.site2.com/v1'.

Cross-Domain requests not setting cookies/getting blocked?
When serving your API from a secure https:// URL and using a self-signed certificate, you may notice weird behaviour in certain browsers. In Chrome, everything was OK, except for the fact that cookies set by my django views, although visible in the response, were being ignored. As for Firefox, POST requests were being blocked by the browser. Using a proper certificate fixed these issues. Just an FYI.

Navigation elements breaking in IE?
Lack of support for the HTML5 history API broke navigation on IE. Use something like the snippet below to render href attributes for <a></a> elements in your angular templates. I also attach the hashURL service to $rootScope so that it's available in all my templates.
module.run(function(..., hashURL , ...){
    $rootScope.hashURL = hashURL;
}).factory("hashURL", function(URL, $location){
    // Used to build hashbanged urls if the HTML5 history.pushState
    // API is unavailable.
    var hashURL;
    if(!$location.$$html5){
        hashURL = URL;
    } else {
        hashURL = function(name){
            return "#" + URL(name);
        };
    }
    return hashURL;
});
and in your templates, use it like this
View more <a href="{{ hashURL('products') }}">products</a>.
Now whether you decide to support IE or not, I still think it's a good idea to refer to URLs by name using a service in your templates.

Serving from localhost:8000 breaking ngResource?
In Angular, the : symbol is used to define replaceable components that are part of URLs in ngResource/djResource for eg., http://example.com/api/products/:id. You need to escape the ":" in URLs passed to these services. I use a URLParse service (borrowed from here) to parse the port number out of the URL to escape the ":" before it. Note, URLs passed to $http don't need this escaping.
module.factory("URLParse", function($window){
    var parser = document.createElement('a');
    return function(url){
        parser.href = url;
        return {
            href: parser.href,
            protocol: parser.protocol,
            host: parser.host,
            hostname: parser.hostname,
            port: parser.port,
            pathname: parser.pathname,
            hash: parser.hash,
            search: parser.search,
        };
    };
}).factory("ResourceURL", function(URL, URLParse){
    return function(name){
        var url = URL(name),
            port = URLParse(url).port;
        return url.replace("\:" + port, "\\:" + port);
    };
});

Sunday, June 22, 2014

$locationChangeSuccess not fired in service

angular.module("app", [
).factory("TrackerUtil", function($rootScope, $location){
    var visited = [];
    $rootScope.$on('$locationChangeSuccess', function (event, toState, fromState){
        visited.push(toState);
    });
    return {
        getVisited: function(){
            return visited;
        }
    };

}).controller("trackedListCtrl", function($scope, ..., TrackerUtil){
    ...
    $scope.visited = TrackerUtil.getVisited();
    ...
})

So you've got code that looks something like this. You're hoping every time the location changes (somebody clicks a link, etc), visited.push() will get called. But for some reason, your service behaves very erratically - fires only sometimes, fires only for certain views, etc. So what went wrong?

When TrackerUtil is registered as an angular service, it doesn't really do much, that is unless you are actually referencing it somewhere in your code like from a controller or from another service. Only if this is the case does angular actually go execute the service body - in our case, registering a listener for the $locationChangeSuccess event. If the service isn't used anywhere, angular doesn't even try checking for it's dependencies.

In my case, I was calling TrackerUtil.getVisited() only from trackedListCtrl and hence had added TrackerUtil as a dependency only for that controller. Because of this, visited was always empty until a view handled by trackedListCtrl was activated, giving the appearance of erratic behaviour.

Solution?
I added it to the app's run() section. Not a great solution, but it works.

Tuesday, March 4, 2014

Error starting django manage.py shell / runserver

So you're hacking away some django code for your next project and decide to test it out on runserver or in django's shell. But then you see this Error: No module named <module> - Not very helpful is it?

Next time this happens, try running runserver/shell (actually any django management command) with the --traceback option to figure out what is really going on.
./manage.py shell --traceback

Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\django\core\management\base.py", line 217, in execute
    translation.activate('en-us')
  File "c:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 105, in activate
    return _trans.activate(language)
  File "c:\Python27\lib\site-packages\django\utils\translation\trans_real.py", line 194, in activate
    _active.value = translation(language)
  File "c:\Python27\lib\site-packages\django\utils\translation\trans_real.py", line 183, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "c:\Python27\lib\site-packages\django\utils\translation\trans_real.py", line 160, in _fetch
    app = import_module(appname)
  File "c:\Python27\lib\site-packages\django\utils\importlib.py", line 35, in import_module
    __import__(name)
...
...
...
  File ".\project\models.py", line 2, in <module>
    from csv import DictWriter
ImportError: cannot import name DictWriter
models.py in the project app is importing the csv library which is supposed to be a standard library shipped with python. So now why is it missing features all of a sudden?
>>> import csv
>>> csv
<module 'csv' from 'csv.py'>
>>> import socket
>>> socket
<module 'socket' from 'c:\Python27\lib\socket.pyc'>
Importing csv.py in a separate python shell (not django shell) in the same folder revealed the cause - csv.py was being imported from the root of the project directory and not the usual place (C:\Python27\Lib\ on a windows machine). This one turned out to be just a dummy script that I was working on and for lack of a better name, just decided to call it csv.py. When manage.py scripts are running, it loads models.py's from all installed apps and in my case, one of them inadvertently loaded my custom csv.py, thus overriding the actual python library. This resulted in an ImportError in my case - could even  be something else like an AttributeError for you.

Errors like this can also happen because of cyclic imports (A imports B, B imports C, C imports A). To fix these, look at the traceback, figure out where the cycles are happening, and remove the problematic imports from the __main__ of the file - move these to wherever you are actually using it (like within a function). The interpreter will try running import statements inside functions only upon function invocation and not when the file is being read (unlike import statements in __main__/imports outside function bodies).
from app1.models import ModelA
def view1(request):
    return ModelA.objects.filter(user=request.user)
Instead, rewrite it like this.
def view1(request):
    from app1.models import ModelA
    return ModelA.objects.filter(user=request.user)

You don't have to worry about the performance penalty about re-importing the same module again and again because python loads modules and keeps them in memory inside a dict sys.modules, meaning the next time you try to import the same module, it sees that it's already got a copy with itself and doesn't repeat work.

window.ready vs window.load vs jQuery.ready() / DOMContentLoaded

DOMContentLoaded AKA jQuery.ready() is fired when the browser is done parsing the DOM, which includes script elements and iframes.

The load event is fired after DOMContentLoaded is fired. At this point, all resources are downloaded and ready which includes frames, images, etc. The browser's stop button stops spinning at this point.

index.html
<html>
    <head>
        <script type="text/javascript">
            console.log("HEAD_SCRIPT_INIT");
        </script>
    </head>
    <body>
        <script type="text/javascript">
            console.log("SCRIPT_TAG_1_INIT");
        </script>
        <script type="text/javascript">
            console.log("SCRIPT_TAG_2_INIT");
            document.addEventListener("DOMContentLoaded", function(){
                for(var i=0; i<10; i++)
                    console.log("SCRIPT_TAG_2_DOM_READY", i);
            });
            addEventListener("load", function(){
                console.log("SCRIPT2WINDOWLAOD");
            })
        </script>
        <iframe srcdoc='
            <script type="text/javascript">
                for(var i=0; i<10; i++)
                console.log("IFRAME_1_INIT:", i);
            </script>'>
        </iframe>
        <iframe srcdoc='
            <script type="text/javascript">
                for(var i=0; i<10; i++)
                    console.log("IFRAME_2_INIT:", i);
            </script>'>
        </iframe>
        <iframe src="iframe_3.html"></iframe>
        <iframe srcdoc='
            <script type="text/javascript">
                console.log("IFRAME_4_INIT");
                document.addEventListener("DOMContentLoaded", function(){
                    console.log("IFRAME_4_DOM_READY");
                })
                window.onload = function(){
                    console.log("IFRAME_4_WINDOW_ONLOAD");
                }
            </script>'>
        </iframe>
        <script async type="text/javascript" src="script_3.js"></script>
        <script type="text/javascript" src="script_4.js"></script>
        <script type="text/javascript">
            console.log("SCRIPT_TAG_5_INIT");
        </script>
        <script type="text/javascript">
            var script = document.createElement('script');
            script.innerText = 'console.log("HEAD_APPENDCHILD_INIT")';
            document.getElementsByTagName('head')[0].appendChild(script);
        </script>
        <script type="text/javascript">
            addEventListener("load", function(){
                console.log("SCRIPT7");
            })
        </script>
    </body>
</html>

iframe_3.html
<script type="text/javascript">
    console.log("IFRAME_3_INIT");
    document.addEventListener("DOMContentLoaded", function(){
        console.log("IFRAME_3_DOM_READY");
    })
    window.onload = function(){
        console.log("IFRAME_3_WINDOW_ONLOAD");
    }
</script>

script_3.js
console.log("SCRIPT_TAG_3_INIT");

script_4.js
console.log("SCRIPT_TAG_4_INIT");

Now, open index.html in a browser. Now open console and draw the following conclusions for yourself from our experiment.

As the downloaded html is parsed by the browser from top to bottom, any script tags in the head are executed followed by any script tags or iframe tags in the body.

Whether a script is specified inline or with an 'src' attribute, execution is done in order and it blocks parsing of the DOM unless the 'async' attribute is specified. 'async' scripts are downloaded out of order with respect to the other resources in the DOM. Downloading them will not block parsing the rest of the DOM, but they will block the load event during execution (it doesn't block DOMContentLoaded as far as I know).

Any script tags within iframe tags are parsed and executed right there if the content of the iframe is specified inline. If the iframe has an 'src' attribute, parsing of the DOM isn't blocked and the resource is downloaded in the background and executed when it's ready. Else it blocks further parsing. Also they fire their own DOMContentLoaded and load events and thus block the parents DOMContentLoaded and load events.

Iframes do not execute in parallel.

No script is EVER interrupted and execution is not switched to a different script just because an event was fired. The firing of every event blocks until the current block is done executing.

A script appended to head via JS code will be executed immediately (just like inline JS).