Issue
I want to make a CGI that launches a program and displays the output in the web-browser in real-time. Real-time here means as soon as a line of output is produced by the program it should be displayed on the browser page.
I choose to write it in bash to wrap the program execution so I can process request parameters.
webbrowser -> nginx+fastcgi -> bash -> program
I have a testing program that outputs one line every half a second, 10 times.
I thought I could declare a plain text context type in response header and then exec the program.
Unfortunately the output appears in the browser only at the end of the execution all at once. I tested it in Firefox and curl.
I have tested many options and any combination of them to workaround the issue :
- disable buffering in nginx with
fastcgi_buffering offdirective - add
X-Accel-Bufferingheader - use
stdbuf -oL program - use xml http request (XHR) + server side event (SSE) instead of plain text.
Nothings works.
I guess the buffering issue is between nginx and bash but I find no way to disable it.
What can I try ?
Solution
I found no way to fix buffering issue in my chain. I tried perl instead of bash with no luck.
So I choose to fill the buffers : after each line of output of the controlled program I echo a bunch of '\0'. Since this content can not be process as a plain text by the web browser, I use the server sent event approach.
#!/bin/sh
printf "HTTP/1.0 200 OK\r\n"
printf "Content-type: text/event-stream\r\n"
printf "Cache-Control: no-cache\r\n"
printf "X-Accel-Buffering: no\r\n"
printf "\r\n"
flush() {
padding=4100
dd if=/dev/zero bs=$padding count=1 2>/dev/null
}
subprogram | while read l;
do
printf "data: ${l}\n\n"
flush
done
The wrapping page looks like that :
<html>
<head>
<meta charset="UTF-8">
<title>Server-sent events demo</title>
</head>
<body>
<pre></pre>
<script>
var evtSource = new EventSource('/sse.sh?subprogram');
var pre = document.querySelector('pre');
evtSource.onmessage = function(e) {
pre.textContent += e.data + '\n';
}
</script>
</body>
</html>
The web browser, in that case, takes care of removing extra '\0'.
The drawback is that the cgi output is far larger than the program output.
Answered By - Setop Answer Checked By - Robin (WPSolving Admin)