https://prefect.io
Join Slack
Hello everyone, there's a way to configure Prefect so he can use a proxy server? I'm trying to log i...
i

Italo Barros

about 4 years ago
Hello everyone, there's a way to configure Prefect so he can use a proxy server? I'm trying to log in on Prefect cloud (using the CLI), but I'm receiving the following SSLError:
requests.exceptions.SSLError: HTTPSConnectionPool(host='<http://api.prefect.io|api.prefect.io>', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:852)'),))
i
k
  • 2
  • 27
  • 341
Hi! Thanks for Prefect! I was wondering when we could expect map support in Prefect 2? No worries if...
t

Tim-Oliver

over 3 years ago
Hi! Thanks for Prefect! I was wondering when we could expect map support in Prefect 2? No worries if this can't be answered. Just curious!
✅ 1
t
k
+3
  • 5
  • 67
  • 340
<@ULVA73B9P> I get this error when trying to set my Prefect profile. I use Prefect 2.10.3: ```Unknow...
h

hagai

about 1 year ago
@Marvin I get this error when trying to set my Prefect profile. I use Prefect 2.10.3:
Unknown setting 'PREFECT_SERVER_ALLOW_EPHEMERAL_MODE'. (type=value_error)
When I run
prefect config unset PREFECT_SERVER_ALLOW_EPHEMERAL_MODE
I get the same error
h
m
  • 2
  • 1
  • 339
Hello <@ULVA73B9P>, I have some flow runs that are stuck in cancelling and when I try the following ...
a

Alan

about 1 year ago
Hello @Marvin, I have some flow runs that are stuck in cancelling and when I try the following script, it doesn't actually cancel them in the database, it still shows up! What can I do to remedy this?
from prefect import get_client
from prefect.server.schemas.states import State, StateType
from prefect.server.schemas.filters import FlowRunFilter, FlowRunFilterState, FlowRunFilterStateName, FlowRunFilterStateType
import asyncio

async def get_cancelling_flows():
    async with get_client() as client:
        
        state_filter = FlowRunFilter(state=FlowRunFilterState(type=FlowRunFilterStateType(any_=['CANCELLED'])))
        # Query all flow runs
        flow_runs = await client.read_flow_runs(flow_run_filter=state_filter)
        cancelled_state = State(type=StateType.CANCELLED)

        # Loop through each flow run and change its state to "CANCELLED"
        for flow in flow_runs:
            try:
                await client.set_flow_run_state(
                    flow_run_id=flow.id,
                    state=cancelled_state,
                    force=True
                )
                print(f"Flow run {flow.id} has been set to CANCELLED.")
            except Exception as e:
                print(e)

# Run the async function to get the cancelling flows
asyncio.run(get_cancelling_flows())
a
m
  • 2
  • 3
  • 337
Hi. Looking for some tips on self-hosted authentication / authorization (user logins, roles) when ru...
k

Keith Hickey

about 3 years ago
Hi. Looking for some tips on self-hosted authentication / authorization (user logins, roles) when running our own Prefect Orion (2.0) server. We can't use Prefect Cloud due to compliance issues with using the SaaS offering (despite its hybrid security model), and this seems to be about the only box not checked for what we want the tool to do in standalone form. I don't want to have to open it up and bolt on FastAPI OAuth2 support to the web api. Any other solutions or workarounds to make it so that anyone on the network can't arbitrarily start, stop, modify flows? Maybe there's a way to allow all GET requests and block any POST/PUT/DELETE etc. to effectively make it read-only by default, and writable for privileged users?
✅ 1
🔐 2
k
c
+2
  • 4
  • 3
  • 337
How should web applications (Node.js) listen/subscribe to Prefect workflows completion? I have a ta...
o

Ofir

over 2 years ago
How should web applications (Node.js) listen/subscribe to Prefect workflows completion? I have a task that trains a deep learning model through a Web Application. The “*Start training*” button on the web app sends a
POST /model
REST API call to the Node.js server which then in turn calls Prefect 2.0 using the REST API to start a new run (of an existing deployment). The workflow will train the model, run inference on some data and then persist the output to a database. This is obviously an asynchronous operation that may take a few minutes (or more) to complete. Assuming the workflow succeeded, I would like to notify the users (those who have Chrome opened to my web app) that something has happened, i.e. training completed. How should the Node.js be notified when the flow has finished (either successfully / failed)? Is there like a middleware / RabbitMQ / other message queue that the Node.js app can subscribe to, onto which Prefect publishes event? If not, does Prefect expose other broadcast events? And if not, should I poll periodically from my app and maintain state diff? Thanks!
o
a
  • 2
  • 13
  • 335
Hey guys, I'm kinda struggling to set up a Prefect on our company server. So it's running and all th...
n

Numline1

about 3 years ago
Hey guys, I'm kinda struggling to set up a Prefect on our company server. So it's running and all that, but I had to setup a reverse proxy (Nginx) as well, so that people can access the UI via standard https at 443 and at a company subdomain (internal that is). The UI itself is accessible, but it's trying to make REST API requests to http://127.0.0.1/api/something which is obviously not working. Is there any env setting I can use to make it understand it should use https://prefect.internal.company.com/api instead? I couldn't find anything on that topic anywhere 😞 Thanks!
n
r
  • 2
  • 9
  • 335
<@ULVA73B9P> how many tasks can be created/ran at once without causing the postgres sql connection p...
m

Maity

about 2 years ago
@Marvin how many tasks can be created/ran at once without causing the postgres sql connection pool to reach its limit?
m
m
  • 2
  • 5
  • 333
Hello, I am new to prefect and investigating several use-cases to assess if Prefect would be a good ...
v

Valentin Baert

over 3 years ago
Hello, I am new to prefect and investigating several use-cases to assess if Prefect would be a good fit in the toolbox to build a team of engineers dealing with data integration between our company's API and our customers systems. For some use-cases we have to periodically pull files from a remote SFTP server, do some manipulations with the file and some APIs then finally upload a sort of "response" file over the SFTP server. I couldn't find something related to SFTP in the built-in tasks list https://docs.prefect.io/api/latest/tasks/function.html What would be the best way to deal with SFTP file exchange with prefect ? Should I just use a custom function task and a python sftp library or is there a more suitable task ?
v
a
  • 2
  • 2
  • 333
<@ULVA73B9P> How can I fix the RuntimeWarning: coroutine 'run_flow_async' was never awaited when usi...
s

Steffen

12 months ago
@Marvin How can I fix the RuntimeWarning: coroutine 'run_flow_async' was never awaited when using await in my code?
s
m
  • 2
  • 13
  • 331
Previous111213Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by