no output from sort-bed for large files

[sjn]

OK, thanks.  We'll plan to improve this in v2.4.1.  See https://github.com/bedops/bedops/issues/34

[sjn]

This is functional in our 2.4.1 development version.

git pull
git checkout v2p4p1

then, clean things and rebuild/install.
make very_clean && make && make install

[pachkov]

Hi!

Thank you for this feature. Indeed it keeps temporary files in specified directory!
However I have got problem that at very end it crashes:

*** glibc detected *** ./sort-bed: double free or corruption (out): 0x000000002aa12470 ***
...

The final bed file is fine but in the temporary dir still contains some temp files left (in my case 3 out of 14).
Any suggestions?

Best,

Mikhail

[sjn]

OK, we'll take a look, thanks.  How are you using things?  --tmpdir or with TMPDIR environmental variable?

Shane

[pachkov]

I used --tmpdir. I will test latter $TMPDIR.

BTW, if I can help somehow with tracing down this bug, please tell me. I will be glad to help.

Best,

Mikhail

[sjn]

Okay, I'm seeing a double-free now.  I'll look in detail.

[sjn]

This has been fixed in a commit to version v2p4p1 now.
Feel free to:
git pull
make
make install

and give this a whirl again.  Sorry for the troubles and thanks for reporting the problem.

Shane

[pachkov]

Hi Shane,

Tried v2p4p1. It finished without any error messages, sorted bed looks fine, tmpdir is empty in the end. Now started run without --tmpdir but with set $TMPDIR. It seems to work correctly.

Good work! Thanks!

Best,

Mikhail

[sjn]

Great.
We're getting close to releasing v2p4p1.  I've made some new changes that should make things a bit more robust.  Previously, if you specified a --tmpdir that pointed at a file (not a directory), things would run and give crazy results.  I've cleaned up some other things too.  If you wouldn't mind testing again, that would be greatly appreciated!

There is one more piece that is a bit disturbing.  If something fails (such as running out of disk space in your case), sort-bed won't clean up any generated tmp files for you.  I'm still looking at how that might be accomplished.  It's a pain in C to deal with these sorts of things.  It can be done, of course, but I run the risk of introducing more bugs.  Thinking about how I might robustly test some things.  Anyway, not sure if this will happen or if I'll chalk it up to "well, we keep those files around so you can look over things upon failure"...

Shane

[pachkov]

Hi Shane,

I have downloaded yesterday v2p4p1, compiled and tested sort-bed. It works flawless. I tested botth with --tmpdir and with $TMPDIR.

I agree that garbage left after program crash is a big headache. My only suggestion is to add warning message at the start in case --tmpdir or $TMPDIR is set. Something telling that non standard tmp dir will be used and potentially some files might not be cleaned after program finished.

Best,

Mikhail

[sjn]

Awesome - thank you!

We have officially released version 2.4.1.