With an Express.js (Node.js) + Mongoose (MongoDB) configuration, a problem occurred where batch processing would terminate partway through when running for a long time.

The situation was that even though the cursor should still be able to fetch the next data, the next data wouldn’t come through in stream.on ‘data’, and stream.on ‘close’ would be called without stream.on ‘error’ being called. The cause seems to be that the cursor was timing out.

As a solution, by specifying noCursorTimeout: true in the third argument, I was able to execute batch processing to completion without terminating partway through.

var stream = User.find(
  {},
  {},
  { noCursorTimeout: true }
).stream();

stream.on('data', function (doc) {
  // do something with the mongoose document
}).on('error', function (err) {
  // handle the error
}).on('close', function () {
  // the stream is closed
});

Reference Information

• Mongoose API v3.8.16
• Stream Node.js v0.10.31 Manual & Documentation

That’s all from the Gemba.