I am working through some of the examples from the...
# marvin-aib
Brock
11/29/2024, 3:57 PMI am working through some of the examples from the website, in particular, the Seinfeld example. I copied the code from the website, and the only difference is that I am using an LLM from Vertex AI (gemini-1.0-pro). A few observations:
• From the logs in the shell, I can see situations where the same agent is going, over and over again
• I didn't see this yesterday, but there appear to be summaries of the flow, which was not part of the agents or the task
• In a few test runs today, I let one run today for well over three minutes, and there appears to be this concept of circular references, repetition (multiple summaries/restarting), and the longer it goes, starting to go off the rails (talking about financial advice, which is no where near the topic used)
• If there is a way to share the logs of these flow runs, I am more than happy to provide these to folks who want to explore
Replying in the thread with a couple of examples from the terminal
Brock
11/29/2024, 4:01 PMExample 1, empty agent calls
Copy code
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:59 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:46:00 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:46:01 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:46:03 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:46:04 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:46:05 PM ─╯Brock
11/29/2024, 4:02 PMExample 2: Not part of the topic after the discussion ends
Copy code
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ The discussion demonstrates the ongoing debate surrounding instant replay and its implications │
│ for the future of baseball. It emphasizes the importance of balancing technological │
│ advancements with preserving the core values and traditions of the sport. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:35 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:36 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:37 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:39 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
│ End of Discussion. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:41 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:42 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:44 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:47 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:48 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:49 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:50 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:51 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:52 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:54 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:55 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ This concludes the task. │
│ │
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
│ ┃ Please let me know if you have any other tasks for me. ┃ │
│ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:56 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ I can also perform many other tasks, such as: │
│ │
│ • Writing different kinds of creative text formats, like poems, code, scripts, musical │
│ pieces, email, letters, etc. I will try my best to fulfill all your requirements. │
│ │Brock
11/29/2024, 4:04 PMExample 3 - Summaries and insights which are not part of the prompt
Copy code
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
│ The responses are: │
│ │
│ • Jerry: I'm torn on instant replay. It's good for getting calls right, but it disrupts the │
│ flow and takes away the human element. │
│ • Elaine: Instant replay is like a bad blind date - awkward, frustrating, and unnecessary. It │
│ takes away the drama and excitement of close calls. │
│ • George: Instant replay is great, but it should be used sparingly. It takes away from the │
│ human element and the possibility of miraculous comebacks. │
│ • Kramer: Instant replay is the greatest invention since the self-stirring coffee mug! It │
│ eliminates umpire errors and settles controversial calls. │
│ • Newman: Instant replay is rigged to favor the bigwigs and exploit the little guy. It's │
│ another way for technology companies to make money and for the powerful to manipulate the │
│ game. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:29 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
│ Insights: │
│ │
│ The agents have diverse perspectives on instant replay in Major League Baseball. They │
│ highlight both the benefits and drawbacks of the technology, considering its impact on │
│ fairness, accuracy, flow of the game, and human element. While some find it essential for │
│ achieving accuracy and settling disputes, others view it as an unnecessary disruption that │
│ takes away from the sport's charm and unpredictable nature. Additionally, concerns were raised │
│ about the role of powerful entities and technology companies in shaping the implementation and │
│ application of instant replay. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:31 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ The discussion demonstrates the ongoing debate surrounding instant replay and its implications │
│ for the future of baseball. It emphasizes the importance of balancing technological │
│ advancements with preserving the core values and traditions of the sport. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:35 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:36 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:37 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:39 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
│ End of Discussion. │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:41 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:42 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:44 PM ─╯
╭─ Agent: Elaine ──────────────────────────────────────────────────────────────────────────────────╮
│ │
│ │
╰──────────────────────────────────────────────────────────────────────────────────── 3:45:47 PM ─╯j
Jeremiah
12/02/2024, 3:14 PM
Thanks for sharing this - I'm working on tracking this down
b
Brock
12/02/2024, 3:46 PMOf course. I know it’s a lot of output which might be hard to parse . My account has nothing personal or proprietary so I am happy to share logs or access too.
d
Dave Aitel
12/02/2024, 8:22 PMAh it would be good to know when this is fixed !
3 Views