I have a named pipe in linux and i want to read it from python. The problem is that the python process 'consumes' one core (100%) continuously. My code is the following:
FIFO = '/var/run/mypipe'os.mkfifo(FIFO)with open(FIFO) as fifo:while True:line = fifo.read()I want to ask if the 'sleep' will help the situation or the process going to loss some input data from pipe. I can't control the input so i don't know the frequency of data input. I read about select and poll but i didn't find any example for my problem. Finally, i want to ask if the 100% usage will have any impact on the data input(loss or something?).
edit: I don't want to break the loop. I want the process runs continuously and 'hears' for data from the pipe.
Best Answer
In typical UNIX fashion, read(2) returns 0 bytes to indicate end-of-file which can mean:
- There are no more bytes in a file
- The other end of a socket has shutdown the connection
- The writer has closed a pipe
In your case, fifo.read() is returning an empty string, because the writer has closed its file descriptor.
You should detect that case and break out of your loop:
reader.py:
import osimport errnoFIFO = 'mypipe'try:os.mkfifo(FIFO)except OSError as oe: if oe.errno != errno.EEXIST:raiseprint("Opening FIFO...")with open(FIFO) as fifo:print("FIFO opened")while True:data = fifo.read()if len(data) == 0:print("Writer closed")breakprint('Read: "{0}"'.format(data))Example session
Terminal 1:
$ python reader.py Opening FIFO...<blocks>Terminal 2:
$ echo -n 'hello' > mypipe Terminal 1:
FIFO openedRead: "hello"Writer closed$ Update 1 - Continuously re-open
You indicate that you want to keep listening for writes on the pipe, presumably even after a writer has closed.
To do this efficiently, you can (and should) take advantage of the fact that
Normally, opening the FIFO blocks until the other end is opened also.
Here, I add another loop around open and the read loop. This way, once the pipe is closed, the code will attempt to re-open it, which will block until another writer opens the pipe:
import osimport errnoFIFO = 'mypipe'try:os.mkfifo(FIFO)except OSError as oe:if oe.errno != errno.EEXIST:raisewhile True:print("Opening FIFO...")with open(FIFO) as fifo:print("FIFO opened")while True:data = fifo.read()if len(data) == 0:print("Writer closed")breakprint('Read: "{0}"'.format(data))Terminal 1:
$ python reader.py Opening FIFO...<blocks>Terminal 2:
$ echo -n 'hello' > mypipe Terminal 1:
FIFO openedRead: "hello"Writer closedOpening FIFO...<blocks>Terminal 2:
$ echo -n 'hello' > mypipe Terminal 1:
FIFO openedRead: "hello"Writer closedOpening FIFO...<blocks>... and so on.
You can learn more by reading the man page for pipes:
- PIPE(7) - Linux Programmer's Manual
- FIFO(7) - Linux Programmer's Manual
(Years later) If I'm understanding the OP's use case using for ... in ... does exactly what is desired:
import osFIFO = 'myfifo'os.mkfifo(FIFO)with open(FIFO) as fifo:for line in fifo:print(line)This program patiently waits for input from the fifo until is is provided, then prints it on the screen. No CPU is used in the meantime.
This is also the more idiomatic way in Python so I would recommend it rather than using read() directly.
If the client side writing to the fifo closes, the for loop ends and the program quits. If you wanted it to reopen the fifo to wait for the next client to open it you can put the for section into a while loop:
import osFIFO = 'myfifo'os.mkfifo(FIFO)while True:with open(FIFO) as fifo:for line in fifo:print(line)This will reopen the fifo and wait as usual.