Buildozer Compiles Apk, But It Crashes On Android

I am able to build an .apk, but after I install it on my android phone it simply crashes at startup. My thoughts for failing is that I am using 3rd party libraries e.g(beautifulsou

Solution 1:

I ran into this problem as well, but I was (apparently) able to get everything working fine with a workaround. Since it doesn't look like you posted a logcat, I'll assume you ran into the same issue I did.

Yes, you do need to list beautifulsoup4 as a requirement in your spec. From looking into bs4's code, it looks like bs4 is willing to use any of several "builders." It supports HTMLParser, html5lib, or lxml. I have no idea why we can't load HTMLParser, but it's actually the least preferred library of the three, and if it weren't for the fact that there's no try block around the import, it seems that everything would work fine (as long as one of the other parsing libraries was available).

With this in mind, I included one of the other libraries, and I decided to hack the import process so that Python would pretend _htmlparser loaded okay :)

This article was instructive: http://xion.org.pl/2012/05/06/hacking-python-imports/

The end result was something like this:

import imp
import sys

classImportBlocker(object):

    def__init__(self, *args):
        self.black_list = args

    deffind_module(self, name, path=None):
        if name in self.black_list:
            return self

        returnNonedefload_module(self, name):
        module = imp.new_module(name)
        module.__all__ = [] # Necessary because of how bs4 inspects the modulereturn module

sys.meta_path = [ImportBlocker('bs4.builder._htmlparser')]
from bs4 import BeautifulSoup

I also added html5lib to the requirements in buildozer.spec.

Now, is this the right way to solve the problem? I don't know. The best approach would probably be to request that the author fix it. It might be as simple as to put the import in a try block. Nevertheless, this is the approach I've gone with for the moment, and it is at least an interesting exercise, and a way to test your app until a better fix comes along.

Additionally, I should warn you that I only did this recently, so I can't 100% guarantee that it won't cause any problems, but at a minimum, it's worked well enough to get my app running and scraping the particular website I was interested in. Good luck!

Solution 2:

I understand that this question is some years old. I think the answer @Will gave was fantastic but unfortunately, in my case, I couldn't use it because html5lib was too slow for what I was making. So this is for reference; for anyone who absolutely must use the built-in parser. It's not pretty but it's fairly manageable.

The issue

After much investigation, I nailed down the cause of the problem. In the buildozer log, I noticed that there was a problem compiling the _htmlparser file with the log reading as follows (with my project path replaced with <project-path>):

Compiling <project-path>/.buildozer/android/platform/build/dists/mypackage/private/lib/python2.7/site-packages/bs4/builder/_htmlparser.py ...
SyntaxError: ("(unicode error) \\N escapes not supported (can't load unicodedata module)", ('<project-path>/.buildozer/android/platform/build/dists/mypackage/private/lib/python2.7/site-packages/bs4/builder/_htmlparser.py', 135, None, 'data = u"\\N{REPLACEMENT CHARACTER}"\n'))

Because it was failing to compile, it wasn't being included in the built apk file. So I looked at the file and at the line that was causing the problem: data = u"\N{REPLACEMENT CHARACTER}" which should actually be replaced by data = u"\ufffd"

Quick-fix

Now the easy way out would be to modify the file right in the package. You could just edit the file mentioned in the error above which should, in theory, work but it's not recommended because every time the package gets re-installed or the code gets built from another machine the problem will be back.

Marginally better fix

It would be nice if the fix was all packaged into our code, so drawing inspiration from @Will's answer, here is the code you would need to put in before your bs4 import:

import sys

classImportFixer(object):
    def__init__(self, mname):
        self.mname = mname

    deffind_module(self, name, path=None):
        if name == self.mname:
            return self
        returnNonedefload_module(self, name):
        import _htmlparser as module
        module.__name__ = name
        return module

sys.meta_path = [ImportFixer('bs4.builder._htmlparser')]
from bs4 import BeautifulSoup

The main difference is that you need to copy the _htmlparser.py file from the bs4 package to your current directory and fix the before mentioned line data = u"\N{REPLACEMENT CHARACTER}" with data = u"\ufffd"

Then, every time the bs4 module is imported, the import is magically intercepted and the local file is used.

Warning: if you ever update Beautiful Soup, you may need to use a more recent _htmlparser.py file in your project with the same fix made as needed.

Final comment

If I made some typos, grammar mistakes or didn't make sense at all, keep in mind that I worked most of the weekend to fix this and quite frankly I'm not thinking straight. Comment if you have any questions and I'll reply and/or edit my answer when I get a chance.