memory monitoring

esalinasesalinas BroadMember, Broadie ✭✭✭

I ran an analysis using a vcf with 2.2G. Initially, I specified in WDL to use 10 G, and the workflow was failed. However, when I specified to 20G, the workflow successfully ran.

Is there anyway that I can monitor how much memory I used or I need during the analysis?

Tagged:

Best Answer

Answers

  • esalinasesalinas BroadMember, Broadie ✭✭✭

    An idea is to use the free command.

    A log could be captured in a file like so.


    MEM_DAT=mem.log ; MEM_FILE=/var/run/free.run ; touch $MEM_FILE ; while [ -f "$MEM_FILE" ] ; do date ; free ; sleep 1 ; done > $MEM_DAT & .... do bioinformatics here.... .... rm -v $MEM_FILE ;

    and the output block could include mem.log.

  • abaumannabaumann Broad DSDEMember, Broadie ✭✭✭

    Currently Eddie's suggestion is your best bet. Long term Google will be providing better monitoring tools on the status of each task that is run, which would support this.

Sign In or Register to comment.