clarification of 'split' event

[dominictarr]

I'm not sure from your readme what the split event emits?

does it emit the buffer up to the just before the token starts?

also, does it join buffers together if they do not already end in a token?

in other words, can I use this to rechunk a stream so that the chunks always break on newlines, for example?

[dodo]

I'm not sure from your readme what the split event emits? you can specify more than one split token .. so it's emitted whenever a token is found.

does it emit the buffer up to the just before the token starts? yes.

also, does it join buffers together if they do not already end in a token? when size is flexible it joins everything together what it gets to one buffer (accessible through stream.buffer or stream.getBuffer()) whenever it gets data, it will try to find all tokens

in other words, can I use this to rechunk a stream so that the chunks always break on newlines, for example? yes.

stream = new BufferStream({size:'flexible'});
stream.split('\n', function (line) { // line doesn't have a '\n' anymore
    stream.emit('data', line); // Buffer.isBuffer(line) === true
});

if you have more question, or does anything concerns you about the api, please point them out.

anyways, i'll update the readme :)

[dominictarr]

that is still a little confusing, but I assume that if the raw stream is like this:

"I am a raw stream\n he", "ar me R", "OAR\n", "!!!!!1!!!" 

if you are splitting on '\n', the split event will give you:

"I am a raw stream", "hear me ROAR", "!!!!!1!!!"

• excluding the tokens,
• still emitting the chunks before the first token and after the last token?

is that correct?

the only thing that I think you should add is to go

new BufferStream({size: 'flexible', split: token})

and then have the BufferStream emit the "split" events as "data" events.
this would mean that you could use pipe to send it to another stream.

[dodo]

not quite, "!!!!!1!!!" would be stuck in the buffer because there is no "\n" after it to cut it off.

btw thanks for the idea ... plz moar of that ;)

[dominictarr]

ah, it might be a good idea to also still emit the last thing.

that is usually what you'd want if you are splitting by newlines...

maybe add an {inclusive: boolean} option, and default it to true?

[dominictarr]

another important nuance:

what happens if the stream has split tokens immediately following each other "data\n\ndada"?

in http, a double newline is used to signify the end of the header section.

[dodo]

just discovered, that this allready works :D

so it will emit "!!!!!1!!!" on end.

[dominictarr]

awesome!

On Wed, Sep 21, 2011 at 12:33 PM, ▟ ▖▟ ▖ < [email protected]>wrote:

just discovered, that this allready works :D

so it will emit "!!!!!1!!!" on end.

Reply to this email directly or view it on GitHub: https://github.com/dodo/node-bufferstream/issues/4#issuecomment-2152082

[dodo]

stream = new (require('bufferstream'))({size:'flexible', split:'\n'});
stream.on('data', function (line) {console.log(line.length, line.toString())});
stream.write("data\n\ndada");
stream.end();

results in:

4 'data'
0 ''
4 'dada'

[dominictarr]

that is good.

On Wed, Sep 21, 2011 at 12:42 PM, ▟ ▖▟ ▖ < [email protected]>wrote:

stream = new (require('bufferstream'))({size:'flexible', split:'\n'});
stream.on('data', function (line) {console.log(line.length,
line.toString())});
stream.write("data\n\ndada");
stream.end();

results in:

4 'data' 0 '' 4 'dada'

Reply to this email directly or view it on GitHub: https://github.com/dodo/node-bufferstream/issues/4#issuecomment-2152129

[dodo]

btw .. i'm allready cutting off a http header from a cgi script with bufferstream https://codetu.be/team/codetube/blob/master/src/git-http-backend.coffee#L105

[asnowfix]

It it possible to use this without setting-up the 'split' token? I would like to use at as a Stream interface over a Buffer (or a series of Buffers...), without having to wait for any specific token ('data' event is emitted as soon as there is data in the buffer). Did I knock the wrong door?

[dodo]

yes it is possible. just don't use split :)

[asnowfix]

So this is a bug, because no 'data' event is ever sent without a 'split' set, or at least this is what I have found so far...

[dodo]

where did you found it? have you actually tried it?

[asnowfix]

I would have bothered reporting an issue without having actually tried it. believe it or not ;-)

I am now using https://github.com/samcday/node-stream-buffer, which does the trick, although it is less sophisticated/feature-ruch compared to node-bufferstream.