https://prefect.io
Join Slack
`prefect deployment create` says it should create or update a deployment but it is failing when the ...
# ask-community
a
prefect deployment create
says it should create or update a deployment but it is failing when the deployment exists (prefect 2.0) ?
k
What error do you get and what version are you on?
a
2.0b3
Copy code
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/alexanderbutler/Library/Caches/pypoetry/virtualenvs/analytics-pipelines-nKjHKDtD-py3.9/lib/python3.9/site-packages/prefect/cli/deployment.py", 
line 251, in create
    await spec.create_deployment(validate=False)
  File "/Users/alexanderbutler/Library/Caches/pypoetry/virtualenvs/analytics-pipelines-nKjHKDtD-py3.9/lib/python3.9/site-packages/prefect/client.py", line 82, 
in with_injected_client
    return await fn(*args, **kwargs)
  File "/Users/alexanderbutler/Library/Caches/pypoetry/virtualenvs/analytics-pipelines-nKjHKDtD-py3.9/lib/python3.9/site-packages/prefect/deployments.py", line
280, in create_deployment
    self.flow_storage._block_id = await client.create_block(
  File "/Users/alexanderbutler/Library/Caches/pypoetry/virtualenvs/analytics-pipelines-nKjHKDtD-py3.9/lib/python3.9/site-packages/prefect/client.py", line 856,
in create_block
    raise prefect.exceptions.ObjectAlreadyExists(http_exc=e) from e
prefect.exceptions.ObjectAlreadyExists

Failed to create deployment 'elt-salesforce/primary'
prefect.exceptions.ObjectAlreadyExists
k
I’ll try to replicate one second
Are you running against Cloud 2.0 or local orion?
a
Local
k
I can’t replicate. Could you show me the DeploymentSpec? Just omit if you have sensitive info
a
np, its very barebones so should be easy to replicate in theory
Copy code
from datetime import timedelta
from pathlib import Path

from prefect.deployments import DeploymentSpec
from prefect.orion.schemas.schedules import IntervalSchedule, CronSchedule
from prefect.flow_runners import SubprocessFlowRunner, DockerFlowRunner


FLOW_DIR = Path(__file__).parent / "flows"

# Salesforce ELT
DeploymentSpec(
    flow_location=str((FLOW_DIR / "salesforce.py").absolute()),
    name="primary",
    schedule=IntervalSchedule(interval=timedelta(minutes=5)),
    tags=["pipeline"],
    flow_runner=SubprocessFlowRunner()
)
deployment
Copy code
from prefect import flow, task


@task(name="extract", description="This task triggers an airbyte extraction job for Salesforce data to BigQuery.", tags=["airbyte", "extract"])
def extract():
    print("Data moved.")

@flow(name="elt-salesforce")
def elt():
    extract()
flow
k
Yikes your code worked for me. I’d need to ask a colleague tom since we’re at end of day here. I’ll leave a message and get that to you
Wait what storage are you using? Local?
a
Local yes
k
Ok will leave I message. I was on Local too. I was wondering if you were on a storage that was blocking overwriting
a
Oh also the whole deployment was done via 2 stage docker so it could be a permission thing
k
Could you give me more details on that? What do you mean by 2 stage docker? 😆
I was debugging the issue by Ryan below yours and I found how to replicate
a
Does it seem easily remediable? Curious on RC. 2 stage docker uses a builder image, then a deployment image. Actually my deployment pattern for Prefect is pretty cool. I'll share when back at PC
k
If not remediable by a setting we expose, it might be remediable by a guide how to do it. Will update you tom
Hey @Alexander Butler, I replicated this and I tried it again in front the team and it worked. It seems this weirdness might be related to Prefect version and the database version. Resetting the database or having consistent Prefect versions might get you past this but it was able to work!
and expected to be working
33 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by