holger brandl
05/29/2021, 7:41 PMСергей Романкевич
06/02/2021, 1:35 AMChohang Ng
06/02/2021, 4:03 PMChohang Ng
06/03/2021, 4:49 PMChohang Ng
06/03/2021, 10:07 PMFlorian Kühnlenz
06/07/2021, 6:15 AMIsmail Cenik
06/08/2021, 10:29 PMDahn Jahn
06/09/2021, 7:39 AMinput_filepath
-> function
-> output_filepath
.Carlos Gutierrez
06/11/2021, 10:48 AMRuslan
06/16/2021, 1:46 AMjcozar
06/16/2021, 7:46 AMmichael.urrutia
06/16/2021, 2:26 PMDiego Piloni
06/17/2021, 11:46 AMTim Enders
06/22/2021, 4:34 PMMatheus Cruz
06/23/2021, 3:00 PMRuslan
06/23/2021, 4:39 PMJeffrey Lai
06/24/2021, 12:32 AMSerdar Tumgoren
06/27/2021, 1:02 AMtrue
-> false
or vice versa, the UI appears to be converting them under the hood to strings ("true"
or "false"
, as the case may be). This causes conditional logic that checks for “truthiness” in my flow to behave incorrectly (i.e. conditionals expecting a False
always evaluate as True
since non-empty Python strings always evaluate as True
). I verified this by toggling the values in Inputs UI and switching to the JSON configuration view of the data. I can get the Flow to run correctly by switching to the JSON configuration and correcting the issue, but figured that perhaps I’m missing something basic about the UI. Anyone have advice?Ruslan
06/28/2021, 8:07 PMAusten Bouza
06/28/2021, 8:31 PMLaura Vaida
06/29/2021, 7:21 AMRuslan
06/29/2021, 8:59 AMRuslan
06/29/2021, 2:47 PMNicholas Chammas
06/30/2021, 6:15 PMDatabricksRunNow
tasks that run Databricks jobs. The tasks take parameters, which I provide via Prefect’s Parameter
construct.
These parameters that are fed into Databricks tasks always show up in the UI with these extra nodes for List
and Dict
. I don’t understand what they’re for, or why I need to see them in the UI, and they really clutter things up.
Is this something I can eliminate by restructuring my flow somehow? Or is there some part of Prefect that needs to be changed to eliminate these noisy nodes from the schematic? Or are these List
and Dict
nodes actually useful for something?Thomas Hoeck
07/01/2021, 8:13 AMSean Talia
07/01/2021, 6:28 PMLaura Vaida
07/02/2021, 10:56 AMFor convenience, you can use the snowflake.sqlalchemy.URL method to construct the connection string and connect to the database. The following example constructs the same connection string from the previous example:
from snowflake.sqlalchemy import URL
from sqlalchemy import create_engine
engine = create_engine(URL(
account = 'myorganization-myaccount',
user = 'testuser1',
password = '0123456',
database = 'testdb',
schema = 'public',
warehouse = 'testwh',
role='myrole',
))
I want to create an engine with passing the input for URL with **
instead of putting every variable
but I dont know how to write the input, it should be a str format
that's what I tried, maybe you have an idea how input data should look like
###selecting order data
@task(log_stdout=True)
def create_engine(snowflake_salesforce):
config = configparser.ConfigParser()
engine=create_engine(URL(**config[snowflake_salesforce]))
with Flow('UWG-Mail') as flow:
snowflake_credentials=PrefectSecret("Snowflake")
connection=create_engine(snowflake_salesforce=snowflake_credentials)
but resulting in this error
[12:03 Uhr] Unexpected error: TypeError("unhashable type: 'dict'")
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 863, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/executors.py", line 298, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "<input>", line 36, in create_engine
File "/usr/local/lib/python3.8/configparser.py", line 959, in __getitem__
if key != self.default_section and not self.has_section(key):
File "/usr/local/lib/python3.8/configparser.py", line 668, in has_section
return section in self._sections
TypeError: unhashable type: 'dict'
peter zhao
07/02/2021, 3:35 PM$prefect server start
Pulling postgres ... done
Pulling hasura ... done
Pulling graphql ... done
Pulling apollo ... done
Pulling scheduler ... done
Pulling ui ... done
Creating network "prefect-server" with the default driver
Creating t_postgres_1 ... done
Creating t_hasura_1 ... done
Creating t_hasura_1 ... error
ERROR: for t_hasura_1 Cannot start service hasura: driver failed programming external connectivity on endpoint t_hasura_1 (d6f6316d7f501eb40399dccdacd6bce8c09e565a6057ec6c6d5433274be1f707): Bind for 0.0.0.0:3000 failed: port is already allocated
ERROR: for hasura Cannot start service hasura: driver failed programming external connectivity on endpoint t_hasura_1 (d6f6316d7f501eb40399dccdacd6bce8c09e565a6057ec6c6d5433274be1f707): Bind for 0.0.0.0:3000 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
on docker here's the error message for that container t_hasura_1:
t_hasura_1 exited with code 128
peter zhao
07/02/2021, 3:49 PMapollo_1 | 2021-07-02T15:47:56.760Z {"message":"Cannot query field \"api\" on type \"Query\".","locations":[{"line":2,"column":3}],"extensions":{"code":"GRAPHQL_VALIDATION_FAILED"}}
When I access the gui, website partially loadsBruno Murino
07/02/2021, 4:24 PM