Jon
10/27/2022, 6:49 PMTypeError: not all arguments converted during string formatting
is there a way to get the parameter's value without giving function with a task decorator?Nate
10/27/2022, 6:54 PMprefect.Parameter
is a special type of task
object, it must be used in a flow context - so this approach may not work
what are you trying to accomplish by passing parameters to regular python functions?Jon
10/27/2022, 6:55 PM@task
def get_login_credentials():
...
@task
def extract_with_headless_browser(login_credentials):
... some logic to spin up a browser ...
# not a task, just some imported fn
page_logged_in = browser_utils.login(
page,
LOGIN_URL,
USERNAME_LABEL,
PASSWORD_LABEL,
SIGN_IN_BUTTON_TEXT,
login_credentials["username"],
login_credentials["password"],
)
... scrape something ...
with Flow() as f:
credentials = get_login_credentials()
extract_with_headless_browser(credentials)
browser_utils.login
would have a task decorator, but doesn't seem we can serialize a headless browser 😅Nate
10/27/2022, 7:02 PMcredentials
is some dict-like return value of get_login_credentials
, which becomes login_credentials
in the scope of extract_with_headless_browser
, which looks totally fine to me
can you show the stack trace? I'm wondering which line actually threw that errorJon
10/27/2022, 7:03 PMTask 'extract_electricity_amr_reports': Exception encountered during task execution!
Traceback (most recent call last):
File "/Users/jonyoung/Dev/workflows/.venv/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "/Users/jonyoung/Dev/workflows/.venv/lib/python3.9/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "<string>", line 39, in extract_electricity_amr_reports
File "/Users/jonyoung/Dev/workflows/workflows/common/browser_utils.py", line 81, in login
<http://logger.info|logger.info>(username, password)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 1446, in info
self._log(INFO, msg, args, **kwargs)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 952, in handle
self.emit(record)
File "/Users/jonyoung/Dev/workflows/.venv/lib/python3.9/site-packages/prefect/utilities/logging.py", line 184, in emit
msg = self.format(record)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 927, in format
return fmt.format(record)
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 663, in format
record.message = record.getMessage()
File "/Users/jonyoung/.pyenv/versions/3.9.13/lib/python3.9/logging/__init__.py", line 367, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
username
and password
Nate
10/27/2022, 7:07 PMfrom prefect import Flow
import prefect
with Flow('f') as f:
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello", "World")
f.run()
File "/Users/nate/opt/miniconda3/envs/prefect-1/lib/python3.8/logging/__init__.py", line 373, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
can you just do <http://logger.info|logger.info>(f"{username=} {password=}")
instead of passing multiple args?Jon
10/27/2022, 7:09 PMNate
10/27/2022, 7:10 PMJon
10/27/2022, 7:12 PM