Stack Exchange
Regex error in cloudbuild.yaml file while passing project-id
I'm trying to run a dataflow job using cloud build
steps:
- name: 'gcr.io/cloud-builders/gcloud'
  args:
  dataflow
  jobs
  run
  google-template-job
  --gcs-location=gs://dataflow-templates/latest/PubSub_Subscription_to_BigQuery
  --parameters=inputSubscription='projects/$PROJECT_ID/subscriptions/messages'
  --parameters=outputTableSpec="$PROJECT_ID:beam_samples.streaming_beam"
  --staging-location=gs://cloudbuild-dataflow-testproject123456789-313307/tmp'
  --region=us-central1


Every time I trigger the build I get the following error
ERROR: (gcloud.dataflow.jobs.run) INVALID_ARGUMENT: The template parameters are invalid.
- '@type': type.googleapis.com/google.dataflow.v1beta3.InvalidTemplateParameters
  parameterViolations:
  - description: 'Unmatched regex: ^projects\/[^\n\r\/]+\/subscriptions\/[^\n\r\/]+$'
    parameter: inputSubscription
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gcloud" failed: step exited with non-zero status: 1

My project id has a ' - ' in it so if I replace the $PROJECT_ID with the value of project id, I still get the same error, is there any workaround for this. I don't have any control over stopping the regex check since it's a google provided template.
How do I get past this

Are you sure of your project ID? Did you not mix with project Name? Can you share your project ID, with dummy value for text?

I tried with project name as well, still the same issue. My project id is  testproject123456789-313307

Got it. It's only a command interpreter issue. If you put single quote, you prevent any evaluation of the inside string.
In you case
--parameters=inputSubscription='projects/$PROJECT_ID/subscriptions/messages'

the value 'projects/$PROJECT_ID/subscriptions/messages' is taken as is and therefore the project ID contain uppercase and underscore, that violate the regex pattern.
Change for double quote and it should work great!
--parameters=inputSubscription="projects/$PROJECT_ID/subscriptions/messages"


I tried this as well, but I'm still receiving the same error. I'm really not sure what I'm doing wrong here, since the gcloud command works fine in sdk but each iteration of inputSubscription throws the same error
