Raspberry PI SpiDev python memory leak

Stuck with a problem in your code? Seek help here.

Raspberry PI SpiDev python memory leak

Postby creeg2 » February 23rd, 2015, 8:56 pm

Hi All, I'm using the Adafruit GPIO SPI library. When I run this code, the memory usage of the thread continually increases. This is a code segment from a larger program but I've isolated the leak to this part. If you look at python debug (pdb), the number of "list" objects increases after every SPI call. I've tried tracking down the source of the leak to no avail. Any advice? Try running this segment yourself and watch the memory usage climb with the linux "top" command. Thanks!

Code: Select all
import Adafruit_GPIO.SPI as SPI

spi = SPI.SpiDev(0,0)

def test():
    while 1:
        a=spi.read(4)

test()
creeg2
 
Posts: 5
Joined: February 10th, 2011, 10:54 pm

Re: Raspberry PI SpiDev python memory leak

Postby bandersnatch » February 24th, 2015, 4:59 pm

Python garbage collection (freeing unused memory) is a hotly discussed topic
(G$$gle "Python garbage collection" if you are interested)

The basic rule is that Python will free the memory assigned to an object when the "reference count"
for the object drops to a value of "0".
The problem is that Python does not immediately release the memory as soon as the "reference count" reaches "0".
The point at which Python decides to do this depends on many factors, and the forums are full of heated discussions
and countless conventional & unconventional methods for forcing immediate garbage collection.

If you just want to eliminate the memory leak then you need to isolate the object that keeps growing in size.
In this line in your test example:
...
a=spi.read(4)
...
BOTH objects "a" and "spi" are always referenced and will never be garbage collected.

One or both of these are causing the memory leak, but which one?
An important difference between "a" & "spi" is that
"a" is a LOCAL variable within the test() function
"spi" is a GLOBAL variable declared outside the test() function

Memory leaks are usually associated with global variables because these remain present
during the entire execution of the program.
One would normally assume that the memory allocated to "a" is re-used each time "a" is assigned
but the implicit type-casting that makes Python so easy to use can sometimes also be subtly misleading.

Just to be sure, first try isolating the leaking object by wrapping the spi.read(4) call inside a function:

Experiment 1: Test memory allocation for "a"
----------------------------------------------
import Adafruit_GPIO.SPI as SPI

spi = SPI.SpiDev(0,0)

def read4Bytes():
localBuf=spi.read(4) # read 4 bytes into a local "list"
return localBuf # return this list on the stack as the result of the function call
# "localBuf" only exists inside this function and is no longer referenced
# when the function exits

def test():
while 1:
a = [] # Initialize "a" as an empty list
a=read4Bytes() # Save the list returned by the function in "a"

test()
----------------------------------------------

Python cracks will probably groan about "unncessary initialization" but the aim is to isolate
the leak by explicitly defining the scope and values of the "a" variable.
If this code no longer leaks then "problem solved" ;^)
If the code still leaks then the spi.read() call of the GLOBAL variable "spi" is assigning new
memory every time it is called.
This might only be the buffer for the results but could also lie somewhere deep in the SPI library code.

The next step would then be to make "spi" a local variable

Experiment 2: Test memory allocation for "spi"
----------------------------------------------
import Adafruit_GPIO.SPI as SPI



def read4Bytes():
spiLocal = SPI.SpiDev(0,0) # Create "spiLocal" for each read call. "spiLocal" only exists within this function
localBuf=spiLocal.read(4) # read 4 bytes into a local "list"
return localBuf # return this list on the stack as the result of the function call
# "localBuf" only exists inside this function and is no longer referenced
# when the function exits

def test():
while 1:
a = [] # Initialize "a" as an empty list
a=read4Bytes() # Save the list returned by the function in "a"

test()
----------------------------------------------
The entire SPI library is only used within the scope of the read4Bytes() function
and Python should free all memory allocated to the SPI library when the function exits.

This should eliminate the leak, but is probably not very efficient.
If this solves the problem then you can either:
- just use the code as it is & get on with the rest of your program
- Examine the SPI library in more detail & look at lots of code examples from other people
to see if there are any special aspects to the SPI library that you don't know about
(SPI library initialization, options etc.)

I hope this helps. Let us know what you find out ;^)))
bandersnatch
 
Posts: 150
Joined: September 17th, 2014, 12:06 pm


Return to Help Me! Software

Who is online

Users browsing this forum: No registered users and 1 guest