https://prefect.io logo
t

Talha

05/04/2021, 10:59 AM
Is it possible in Prefect to invoke R script with-in a task. I have a code in R script and that can only run in the R-studio, I don't want to convert it into python. How can I invoke that R code with prefect. Is there an interface between R and Prefect
s

Stéphan Taljaard

05/04/2021, 12:44 PM
I would try calling that script using python subprocess
👍 1
k

Kevin Kho

05/04/2021, 1:16 PM
Hi @Talha, if you have R on the agent, you can do it with the ShellTask, which is similar to what @Stéphan Taljaard is suggesting.
t

Talha

05/04/2021, 1:18 PM
yeah, I am looking at the shell task documentation, can you please provide an example link,
k

Kevin Kho

05/04/2021, 1:19 PM
Copy code
import prefect
from prefect import Task
from prefect import task, Flow, Parameter
from prefect.tasks.shell import ShellTask

shell_task = ShellTask(
    helper_script="",
    shell="bash",
    log_stderr=True,
    return_all=True,
    stream_output=True,
)

@task
def get_command(param):
    return f"echo '{param}'"

with Flow(name="Test") as flow:
    param = Parameter('param')
    cmd = get_command(param)
    test = shell_task(command=cmd)

flow.run(parameters=dict(param='a'))
t

Talha

05/04/2021, 1:19 PM
Also what do you mean when you say if R is running on my agent ? I am a bit confused
k

Kevin Kho

05/04/2021, 1:20 PM
Use
Rscript file.R
to run the R file.
I mean making sure that R is installed where you are running this
t

Talha

05/04/2021, 1:21 PM
okay. sure
Hi, I am trying run the shell task, It is not working. Is it because I am a windows user. I read that windows have issue with shell task on the github
k

Kevin Kho

05/06/2021, 1:23 PM
Hey, you need bash installed and available in your path.
t

Talha

05/06/2021, 3:43 PM
Hey, I have no idea how to put bash into the path. Can you please guide me
please
Do I need to install Linux into windows ?
k

Kevin Kho

05/06/2021, 3:50 PM
I think you can install git bash and then add it to the PATH variable under System Variables with this
t

Talha

05/06/2021, 4:17 PM
I have added git bash to my Env path
can you please give a simple command to check if my shell task is working.
I am using this and it is failing
may be I am doing it wrong
k

Kevin Kho

05/06/2021, 4:20 PM
Copy code
shell_task = ShellTask(
    helper_script="",
    shell="bash",
    log_stderr=True,
    return_all=True,
    stream_output=True,
)
with Flow(name="Test") as flow:
    test = shell_task(command="echo 'test'")
t

Talha

05/06/2021, 4:24 PM
it works. thanks a lot
👍 1
k

Kevin Kho

05/06/2021, 4:27 PM
Nice!
t

Talha

05/07/2021, 3:26 PM
Hey @Kevin Kho, can I add a specific time in the scheduler of my flow. Like i see how to add an interval schedular
but what if I want to run my flow on exact 09:00 AM
k

Kevin Kho

05/07/2021, 3:26 PM
Use the CronScheduler and specify 9 AM and what days
t

Talha

05/10/2021, 11:54 AM
can i provide parameters to the scheduled tasks through Prefect Server UI
I have scheduled 1-minute interval scheduler, but i am not sure how to pass the parameters to the scheduled tasks
Or do I have to pass the parameters through the code]
k

Kevin Kho

05/10/2021, 1:15 PM
Parameters can be passed through the UI
Is this what you’re looking for?
Normally though they are attached to scheduled. You can go to your scheduled and edit the parameters there
t

Talha

05/10/2021, 2:28 PM
this is the same for the prefect server UI
k

Kevin Kho

05/10/2021, 2:30 PM
Can you edit the parameters attached to schedules?
t

Talha

05/10/2021, 2:31 PM
you can see my task scheduled, by 1 minute interval
but it is not be processed. I have even passed the default parameters as well
k

Kevin Kho

05/10/2021, 2:34 PM
Sorry I’m not understanding here. Are the scheduled flows running late (or not running)? I think already scheduled runs might not be updated if they were scheduled before you changed the parameter. Maybe you can try clearing them?
t

Talha

05/10/2021, 2:42 PM
I have been able to run it using UI
👍 1
may be an error in the python code. I have used the default parameters. I can override the default parameters through UI ? right?
k

Kevin Kho

05/10/2021, 2:43 PM
Yes you can
t

Talha

05/11/2021, 10:55 AM
hi @Kevin Kho I am trying to connect to the prefect server on localhost:8080. But i am working in a firm where we use HTTP and HTTPS proxies. The prefect server doesnot connect if i don't bipass the proxy. a
And when I bypass the proxy the flow does not run, as one of the task is api data fetching which needs Proxies. What is the way around it?
k

Kevin Kho

05/11/2021, 2:25 PM
This might be beyond me. Can you make a new post in the server channel so that I can direct it to someone else on the Prefect team?
t

Talha

05/11/2021, 2:54 PM
I posted in the prefect server group. please direct it. I am really confused about this. #
👍 1
Hi @Kevin Kho I am trying to run my shell command and I am receiving an error stating "Command failed with exit code 4294967295". I have attached the screenshot of the simplest example I am running.
I have bash.exe and git in the path as well
I have done the research and i am still not been able to find the solution so far
k

Kevin Kho

05/26/2021, 5:25 AM
Hey @Talha! I thought this simple flow was working before already? I hvaen’t seen this before. What do you think may have changed? Did you upgrade Prefect maybe? Did you path change from something else?
l

Luciano Fernando Palmeira de Almeida

06/04/2021, 8:59 PM
@Ruan Valente Staffuzza
3 Views