Applications of BEDOPS > sort-bed

no output from sort-bed for large files

(1/6) > >>

pachkov:
Dear All,

I was trying to use sort-bed fro sorting our bed files and encountered strange problem. It works fine for files containing up to 50,000,000 lines but for 60,000,000 return no output. No error messages or other hints, just finished with no output.

This how I run it:

zcat test.bed.gz | sort-bed --max-mem 1G - &> out.bed

Are there limitations for sort-bed on file size? How can I find out what is wrong?

Best,

Mikhail

sjn:
Hi Mikhail, thanks for the post.
No, there shouldn't be a problem with this.  What version of BEDOPS are you using?  Can you post the results of sort-bed --version.  We actually released BEDOPS v2.4 yesterday to the community, and one thing we've improved was sort-bed.  If you're using v2.3 or below, it's possible that you are running into a maximum number of open file descriptors problem with --max-mem 1G.  v2.4 resolves these rare issues.  Can you try to download and use the latest software and report back your findings?

Thanks,
Shane

pachkov:
Thanks for prompt reply, Shane!

Version is 2.4.0. I have downloaded 64bit binaries yesterday.

I must add that I have tried to run it with max mem up to 8Gb and still the same thing - empty output.

Best,

Mikhail

sjn:
OK, well, perhaps there is a new bug.  Is there anywhere you could place this file so that we could download it and investigate?  It's huge, but that would certainly help us figure out the problem.  Is this possible?

Also, if you do not redirect stderr with stdout, I wonder if it will print something to your terminal?  It's doubtful, but might be worth trying since this seems to be an easy problem to repeat.

Are you running the linux binaries or the mac binaries?  I've been able to sort hundreds of millions of rows with the --max-mem flag without issue.  Definitely would like to see your .gz test file.

pachkov:
Here is the link:

http://ismara.unibas.ch/ISMARA/sample_data/tmp.bed.gz

I was redirecting stderr and stdout to separate files in order to catch something, but there is nothing. Not sure if it matters but I run binaries (64bit) on quite outdated Linux version (CentOS 5, I believe).

Thanks for help!

Navigation

[0] Message Index

[#] Next page