Iterating Over Pyodbc Result Without Fetchall()

I'm trying to process a very large query with pyodbc and I need to iterate over the rows without loading them all at once with fetchall(). Is there a good and principled way to do

Solution 1:

Sure - use a while loop with fetchone.

http://code.google.com/p/pyodbc/wiki/Cursor#fetchone

row = cursor.fetchone()
while row isnotNone:
    # do something
    row = cursor.fetchone()

Solution 2:

According to official documentation the cursor is apparently an iterator. Therefore, you shouldn't need to create a custom iterator/generator.

If you are going to process the rows one at a time, you can use the cursor itself as an iterator:

cursor.execute("select user_id, user_name from users"):
for row in cursor:
    print(row.user_id, row.user_name)

Solution 3:

you could also use cursor.fetchmany() if you want to batch up the fetches (defaults to 1 if you don't override it)

http://code.google.com/p/pyodbc/wiki/Cursor#fetchmany