in that case, it seems that your chain is backwards! the first sequence name ("QIUM02000100.1" in your chain) is the "from" and the second ("17.10" in your chain) is the "to". This explains why you are getting no variants lifted over. It might be good to have the tool provide a more helpful error...
The developers got back to me and mentioned that porting CombineVariants to GTAK4 is a work in progress. I have let them know that this is a feature that the users wants and hence will be prioritized.
For now you can use it from GATK3.
I hope this helps.
@biojiangke, You would have to create an intervals list to either include -L or exclude -XL these regions during the HaplotypeCaller step.
Hello @JuliaJoung and @mikedamour -
Permissions for Uploading/Deleting in Google Bucket
You can access data from any bucket from FC as you prefer to do by keeping your .bams in your main billing account bucket. While you were able to previously upload files to buckets, your data might not have been accessed this time for permission reasons due to a change in behavior when service accounts were introduced (a blogpost with details on switch to service accounts: https://software.broadinstitute.org/firecloud/blog?id=11342). The FC project’s service account, generated for each FC project, needs to be added to your GCP bucket’s data for FC to be able to access during workflow. The service account details can be found under your account information on FC.
Connecting FC Billing Project to Google Billing Account
The FC Billing Project connects any Google Billing Account that currently/previously exists to the FC workspace created - it is not another billing account. Please make sure that you have added [email protected] as a Billing Account User in your Google Billing Account before you create the FireCloud Billing Project, after which you can remove the user.
Creating Buckets within FireCloud Billing Project
You can create folders within the google bucket that is automatically generated at generation of the workspace but you cannot currently make another bucket within the FC billing project. However you can access data in any other external Google bucket as long as the correct permissions are granted via the service accounts.
Please feel free to reply back with any further questions.
@JuliaJoung Unfortunately there isn’t a way for us to add the proxy group to your GCP bucket for permissions. However I have listed some steps below if a) your bucket is non FC workspace associated or b) your bucket is FC workspace associated.
a) To add your FC proxy group to your non FC bucket:
1. Copy your proxy email address that can be found in your FireCloud Profile under Proxy Group.
2. Navigate to the GCP console page.
3. Open the IAM page and select your project, of which you are a member, followed by the “Add”.
4. Enter the proxy email address you copied from your FireCloud profile here and choose the permissions. This is where the service account is given the ability to get access to data in your GCP bucket. There should be a specific option to add the user (proxy email address) as a service account user.
Here is a more detailed link with more options on permissions: https://cloud.google.com/iam/docs/granting-changing-revoking-access
b) Permission issues for an FC workspace associated bucket:
A FC workspace bucket should have all necessary permissions upon creation of the workspace. If you are unable to delete/upload to a FC associated bucket, please share your workspace with [email protected] for further investigation by our team.
Please reply with any follow-up questions!
Hello @jgould -
1. "RunningJob" is the indicator to mark Cromwell deciding to run the job and the Google PAPI's first action/event timestamp.
2. “Pulling-image” indicates that the appropriate Docker image is being pulled into the virtual machine and “ok” indicates that it was completed successfully and the workflow will continue.
3. The user is not charged when in the “waiting-for-quota” phase.
The timing diagram is a bit different in its visual interpretation. For example, one might give time stamps for the following steps:
10:14:11 - “pulling-image”
10:15:21 - “ok”
11:14:11 - “running-docker”
However, the diagram fills in the space between timestamps by extending the action tag/name (“ok”) across the time difference. So it would show that “ok” is 10:15:21 through 11:14:11 but what it shows is that “running-docker” didn’t start for an hour after the “ok” action.
Please let me know if you have any further questions!
It seems like the issue may be with the WDL script. Would you mind changing this one line:
..to String interval_list. Cromwell is treating the actual intervals as a file, when its really one interval range stripped from a line in a file.
Are you using preemptible machine for your workflow. The error message you are seeing is indicative of a preemptible VM failure but sometimes it has been seen to occur in non preemptible VM cases - this is a bug in the PAPI v1 that should hopefully be eliminated when upgraded to PAPI v2. As a workaround, please re-try your jobs with the maxRetries runtime attribute as it has seemed to override the bug in other instances of the same error. Additionally, you can try with smaller sample sets to avoid termination.
Please feel free to reach out with any other questions/concerns that you may have.
Hi @apampana. The team hopes to support hail 0.2 by the end of the year.
Please feel free to reach out with any further questions!