Take the 2-minute tour ×

I'm trying to build a web app on App Engine that, at a certain point, I need a function to construct a list property with one member from the distinct groups. I'm able to get this working if I use an Sqlite database and web.py database module to maniputate, I can implement what I need like this:

db = web.database(dbn='sqlite',db=':memory:')
db.query("CREATE TABLE seq_list (seq TEXT, seq_pickle TEXT);")

# do some stuff to the database

def getUnique():
    uniq_entries = db.query("SELECT DISTINCT seq FROM seq_list;")
    if not uniq_sequences:
        for entry in uniq_entries:
            seq_query = db.query("SELECT * FROM seq_list WHERE seq='" + str(entry.seq) +"';")
            seq_obj = pickle.loads(seq_query[0].seq_pickle)

I'm having trouble getting this to work in App engine though.

class SeqObj(db.Model):
    seq = db.TextProperty(required = True)
    seq_pickle = db.TextProperty(required = True)

# do some stuff to the database

def getUnique():
    entries = db.GqlQuery("SELECT * FROM SeqObj")
    entries = list(entries)
    if not uniq_sequences:
        unique_entries = set([entry.seq for entry in entries])

        # this prints the unique entries correctly
        print unique_entries

        for entry in unique_entries:
                 q = "SELECT * FROM SeqObj WHERE seq='%s'"% entry
                 print q
                 seq_query = db.GqlQuery(q)

                 # these are the lines that break it
                 seq_obj = pickle.loads(seq_query[0].seq_pickle)
                 print entry

The try/except is for showing the print statements. This outputs for some test data:

set([u'DXTMT', u'DIXTX', u'XSXDV', u'XI*MT'])

When I don't put everything the try catch blocks, I get

IndexError: The query returned fewer than 1 results

I don't know what's going wrong because I the unique sequences are returned, the queries look like I expect them too, and I can verify that the entries are in the datastore. Any help is appreciated.

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

The problem is that you cannot filter on unindexed properties (TextProperty is unindexed). See https://developers.google.com/appengine/docs/python/datastore/queries#Filtering_on_Unindexed_Properties_Returns_No_Results for details.

Try using a StringProperty instead if your seq field is known to be less than 500 characters.

Also in these cases I always suggest to first run the query in the DataStore viewer. It suggests what indexes you need to run the desired query.

share|improve this answer
Nailed my problem, thanks. Since the data I'm handling are sequences of DNA or a protein, there are use cases in which I would need a field longer than 500 characters. Do you know of a workaround for creating an index fields with long chunks of text? –  Sevenless Jul 31 '12 at 20:02
What you would typically do is to add another field called seq_hash or seq_id that represents the checksum of the field, and search by that field. If you plan to have many seq_objs and not many different seqs you could even have another model called SeqIds, having the fields seq and seq_id. And then your first query would fetch all the different seq_ids from that model, and you would use them to query your SeqObj model. Note also that your current model will fetch most of your entities from the datastore. –  Sebastian Kreft Jul 31 '12 at 21:59
If you had been using NDB it would have given you a useful error message instead of an empty result: "BadFilterError: Cannot query for unindexed property seq" . –  Guido van Rossum Aug 1 '12 at 22:35
add comment

Is this failing on dev_appserver or on the real app engine servers?

If it works on dev_appserver but not on the real thing, then I would guess you may be missing some indexes on the real server.

share|improve this answer
add comment

Your Answer


