Fecha de publicación: 26/07/2024 Fuente: Noticias ICEX Lugar: Fundamentals FundamentalsDeep-dive: Optimizing prompts for AI ActionsPractical examples to generate desired responsesThis guide demonstrates the process of optimizing a prompt for AI Actions feature. By following this real-world example, you'll learn how to craft effective prompts to extract valuable insights from your curated articles.What you'll learnBefore diving into the details, here are the key insights you'll gain from this guide:Setting a goal and working backward from the expected result leads to effective prompts.Iterative refinement is crucial for improving prompt performance.Being specific in our requests avoids confusing the AI due to ambiguity.Allowing the AI some flexibility can lead to richer, more insightful responses.Clear instructions are essential, especially when asking the AI to make inferences or assessments.Testing the prompt on various articles helps ensure its robustness and effectiveness across different contexts.TRY AI ACTIONSNote: For purposes of illustration, we focused these examples on a market intelligence use case. The general guidance applies to other use cases as well.Before you startClarifying objectivesTo illustrate the process, let's use a real user request as an example:As a Feedly user, I would like to use AI Actions for Use Case Detection - Identify/build a list of "Use Cases": Application of a technology in a specific business domain - e.g. "AI for weather prediction" or "Immersive Pilot Training" (using VR...) - so we want to label the tech, business domain and track the examples.Market Intelligence CustomerFrom this request, we can distill the following objectives:Identify and build a list of use casesLabel the technology and business domain for each use caseTrack specific examples of each use caseSelecting appropriate test dataFor an effective iteration process, we'll run prompts on articles with familiar content. This allows us to accurately assess the output quality and quickly identify areas for prompt improvement.Outlining the desired outputWorking backward from the desired output is an efficient technique to build an effective prompt.From the requirement, we understand this is a scenario of a recurring action on each article. We want to extract the same information from each article consistently.We need a list of "Use Cases" with the following details for each:Tech labelBusiness domainExamplesA table format can be useful for organizing this information:The optimization processIteration 1: Initial promptWhile crafting the first prompt, we decided to add more columns to generate a more comprehensive output.Step 1: Crafting the Initial PromptPrompt 1Step 2: Analyzing the ResultsAfter running this prompt, we received the following table:Result 1We analyze the results to look for:Missing informationMisinterpretationsOpportunities for more detailed or relevant outputStep 3: Identifying Areas for ImprovementArticle Reference: Having the article's title readily available would help in quickly assessing the results.Action: Add a column for the article title and URL for quick reference.Business Domain vs. Business Unit: The user expects "Business Domain" to be more specific (e.g., operations, sales) rather than generic (e.g., aviation, hospitality). We should add a separate "Business Unit" column.Action: Introduce a "Business Unit" column with examples to guide the AI.Concerned Parties: The AI returned specific names, but we're looking for more generic values (e.g., "airlines" instead of airline names).Action: Replace "Concerned Parties" with "Key Stakeholders" for clarity, and create separate columns for types and names.ℹ️ Note: Similar to the business domain/unit nuance, the interpretation of "Concerned Parties" can vary. To avoid ambiguity, it's crucial to either adapt to the AI's interpretation or provide a precise definition in the prompt.Multiple Use Cases: Some articles mention multiple use cases, but the AI only provided one or aggregated the details.Action: Add a note to explicitly request all use cases from each article.ℹ️ Note: The AI will typically optimize for a concise response. Among other reasons, generating text is one of the most resource-intensive tasks for an AI, and usually, a short, concise, and aggregated response is sufficient for users. If we require a granular and very detailed response, it's best practice to explicitly state this in the prompt.Iteration 2: Refined promptWe updated our prompt based on the previous observations. To test our refinements effectively, we'll focus on article [3], which contains multiple use cases. This targeted approach helps us evaluate specific improvements and refine our prompt more precisely.Step 1: Implementing improvementsPrompt 2Prompt 2: Initial refinement - Structuring the output and guiding the AIStep 2: Analyzing the New ResultsAfter running the updated prompt, we received more detailed results:Result 2Step 3: Further Optimization OpportunitiesUse Case Name: The AI provided very specific internal code names or project names. For better interpretability, we want more explicit names.Action: Update the prompt to request "a short sentence describing the use case" for the Use Case Name.Flexibility in Column Values: While the AI correctly filled in the columns based on our examples, we want to give it more freedom to come up with appropriate values without being constrained by our examples.Action: Remove specific examples from the prompt to allow for more diverse and potentially insightful responses.ℹ️ Note: The AI often possesses broader knowledge than humans in general. Giving them the freedom to generate results without constraining them with examples of expected output might lead to valuable insights we hadn't considered. This approach leverages the AI's extensive knowledge base and can result in more comprehensive and nuanced responses.Iteration 3: Second round of refinementStep 1: Updating the promptBased on these observations, we made another refinement to the prompt:Prompt 3Prompt 3: Intermediate refinement - Enhancing clarity and flexibilityStep 2: Analyzing the resultsResult 3Step 3: More Observations and ImprovementsKey Stakeholders: Without the example constraints, the AI's definition of stakeholders varied. In the "Stakeholders name" column, we got the name of the project instead of the name of the overall entity implementing the project.Action: Change the column "Stakeholders name" to "The party implementing the use case". This update also inspired us to request information about the solution provider.ℹ️ Note: The variation in stakeholder definitions demonstrates the AI's adaptability rather than inaccuracy. This flexibility can lead to valuable insights, but it also highlights the importance of precise prompting when specific information is required.TRL Evaluation: The AI didn't provide an evaluation of the TRL (Technology Readiness Level), stating that there wasn't enough information in the article. This is because our AI is instructed not to give responses if the information is not available in the articles.Action: To encourage the AI to provide its own evaluation or recommendation based on interpretation, we need to explicitly mention this in the prompt. We'll clarify the TRL column: "Evaluated TRL (Give your assessment if the information is not explicit in the article)"ℹ️ Note: In the previous iterations, the AI gave results for TRL. The inconsistency in TRL evaluations reflects the ambiguity in our prompt rather than a limitation of the AI. By using the term "Evaluated TRL," we left room for interpretation, whether the AI should make its own evaluation or retrieve an evaluation from the articles. This highlights the importance of clear instructions, particularly when asking the AI to make inferences or assessments beyond the explicit content of the articles.Iteration 4: Final promptStep 1: Final refinementsBased on these observations, we made our final refinements to the prompt:Prompt 4Prompt 4: Final refinement - Specifying implementers and encouraging AI evaluationStep 2: Validating the final resultsResult 4Step 3: Assessing the Final PromptTo confirm the effectiveness of our optimized prompt, we tested it on two new different articles.Final resultThe results met all our expectations, providing detailed and accurate information for each use case.ConclusionOptimizing prompts is an iterative process that requires careful analysis and continuous refinement. By following these steps and adapting them to your specific needs, you can create more effective prompts for AI Actions in Feedly. Remember to:Start with a clear understanding of your requirements.Craft an initial prompt based on your desired output.Analyze the results thoroughly, identifying both strengths and areas for improvement.Refine your prompt based on your observations, being specific about what you want.Test your refined prompt and analyze the new results.Continue iterating until you achieve the desired output quality.Allow flexibility in your prompts to leverage the LLM's broad knowledge base, which may lead to unexpected insights.This guide provides a comprehensive learning experience for users looking to optimize their prompts in AI Actions by sharing the detailed observations and logic behind each refinement. Remember that prompt optimization is as much about understanding AI's capabilities as it is about clearly defining your requirements.Try AI ActionsSynthesize, translate, extract data, create reports, summarize, and more.START FREE TRIAL