← LOYTI Newsletter

May 2024 · RETROSPECTIVE EDITION

May 2024: From AI answers to controlled business actions

GPT-4o, Gemini 1.5 Flash and Claude tool use bring new possibilities for multimodal workflows, cost-aware routing and supervised integrations.

May’s AI announcements made the technology feel closer to ordinary business interactions: interpreting a picture, responding quickly and connecting an answer to a software tool. Those possibilities were relevant to customer service and internal operations, but they also raised a practical question. At what point does an assistant stop proposing work and start changing business records? The answer should be deliberate. This edition examines three developments through the requirements of a controlled workflow, distinguishing capabilities available during the month from previews and demonstrations that still needed evaluation.

1. GPT-4o broadens the multimodal conversation

OpenAI introduced GPT-4o on May 13. Text and vision capabilities began rolling out through ChatGPT and the API, while the launch described a broader multimodal direction that included voice and video. Not every demonstrated experience was immediately available through every product. The announcement and its evaluations should therefore be read alongside the specific access a business actually had.

For a company handling visual information, a useful experiment might pair a customer’s description with a photograph or compare a visible document with an existing record. Define the job narrowly. An assistant could propose a classification or draft follow-up questions, while a person decides what the evidence supports. Test ambiguous photographs, cropped labels and requests containing contradictory details. If the workflow creates a customer-facing answer, verify product names, instructions and any promises before release. The point is to reduce preparation effort without making uncertainty harder to see.

Business takeaway: evaluate the input types available in the product you can use. A compelling multimodal demonstration is a starting point for testing your own documents and customer scenarios, not a substitute for acceptance criteria.

Source: https://openai.com/index/hello-gpt-4o/

2. Gemini 1.5 Flash makes model routing a business question

Google introduced Gemini 1.5 Flash on May 14 as an option aimed at speed and efficiency. The announcement positioned it alongside updates to Gemini 1.5 Pro. Both were described in preview, with general availability planned for June. A two-million-token context window for Pro was offered through a waitlist, which should not be confused with standard access for every user.

The operational implication was that a business might use different models for different stages of a workflow. A fast, lower-cost model could be evaluated for classifying routine requests, while more difficult cases are routed elsewhere or reviewed by a person. This approach needs evidence. Create categories for straightforward, ambiguous and incomplete inputs, and test whether the routing decision itself is reliable. A cheap first pass that sends too many mistakes downstream can increase total cost. Include the time spent on corrections and the consequences of a missed exception in the comparison.

Business takeaway: assess cost per correctly completed task, not just the price of a model call. Use preview tools for bounded evaluations and keep a clear distinction between experimentation and a supported production dependency.

Source: https://blog.google/innovation-and-ai/technology/developers-tools/gemini-gemma-developer-updates-may-2024/

3. Claude tool use creates a clearer bridge to software

Anthropic announced general availability of tool use for the Claude 3 family in May. The feature let developers describe tools and have the model produce structured requests to use them, supporting tasks such as information extraction and connections to external systems. The developer’s application still needed to implement and control those actions. A model proposing a tool call is not, by itself, an authorization policy.

That distinction is central to useful business automation. Start with read-only actions, such as looking up a permitted record and preparing a summary. Before allowing writes, decide what the assistant may change, which values must be validated and what requires approval. A process that creates a ticket should avoid creating duplicates when a request is retried. A process that proposes a refund should show the evidence and wait for the appropriate person. Keep an audit trail that connects the original request, the proposed action and the final result.

Business takeaway: connect AI to software through explicit permissions and validation. The value comes from a dependable business process around the tool call, not from giving an assistant unrestricted access to act.

Source: https://claude.com/resources/articles/tool-use-ga

One workflow to map this month

Select a task that moves from an incoming document or request to a system update. Mark where information is interpreted, where rules can be checked automatically and where someone approves the result. That map identifies places AI might help while preserving accountability for the decisions that affect customers and money.

Present-day invitation: Book an appointment at https://loyti.com/book to explore an AI-assisted workflow or custom integration for your business. Brice Prather, President, CEO & Founder of Life On Your Terms, Inc. (LOYTI), can discuss AI consulting, custom software and automation in the context of your actual process and operating goals.

Original sources

  1. Hello GPT-4o · 2024-05-13
  2. Gemini 1.5 Pro updates, 1.5 Flash debut and 2 new Gemma models · 2024-05-14
  3. Claude can now use tools · 2024-05-30