Hi Team, Im using Prefect to coordinate some KDB calls via PyKX. When trying to deploy however I keep running into the issue:

Script at './Script/ibes_pykx.py' encountered an exception: ValueError('signal only works in main thread')

. The code is trying to establish a connection and then run these processes on the connection itself. Any advice would be a HUGE help. Source code is below:

def setup():
    token = subprocess.Popen('\\\\ccl\\data\\extlib\\KDBSecToken.exe', stdout=subprocess.PIPE).communicate()[0].decode('utf-8')
    masterConn = kx.SyncQConnection( host='v-kdbr-01', port=5000, username='cleung', password=token, timeout=3.0 ) 
    user = masterConn('.z.u') 
    print('User = ' + user.py())
    port = masterConn('getProcessClient[`prefect_testing_base;`pykx_test]')
    print('Port = ' + str(port.py())) 
    masterConn.close()
    global conn
    conn = kx.SyncQConnection( host='v-kdbr-01', port=port.py(), username='cleung', password=token) 
    user = conn('.z.u') 
    print('User = ' + user.py()) 

@flow()
def ibes_pykx():
    setup()
    logger = get_run_logger()
    <http://logger.info|logger.info>("Initializing initial arguments")
    dailyDates = conn('dailyDates: .dt.drb[.dt.shiftdateb[exec max date from QModelD;-3];  exec max date from QModelD];')
    logFile = conn('inLogFile:`')

   

    return finishIBES_result

    

if __name__ == "__main__":
    ibes_pykx()

What version of Prefect are you running? I think the changes in 2.8.7 will prevent this error most of the time because your flow will run on the main thread. I did a quick test and got the same error in 2.8.6, but not 2.8.7 or above. See here for more info: https://discourse.prefect.io/t/prefect-2-8-7-is-here-with-enhanced-concurrency-handling-engine-reliability-a-new-flow-run-graph-and-more/2593

Im using Prefect version 2.10.3 unfortunately

Ahh - I was on mobile before and I didn't notice the tasks you were running. I'm not sure if

signal.signal

is being called in your code or in a library you're using, but either way I don't think it can work inside a task because the Prefect engine runs tasks in a new thread.

Hmm Ill keep alookout on that

Hi, I am running into the same issue. Did you find any solution @Charles Leung?