Currently, it's very non-intuitive to submit non-text Parts to the model, unless the user can construct the request to the model with one of those parts attached. For example, an agent that lists a set of images and let's the user indicate an image to be processed by an LLM currently requires a combination of a special tool to set the image reference in the state and a before-model-callback to check the state and add the Part.from_bytes() to the llm_request.contents and clear the state.
If a FunctionTool could return a list of parts, including the Part.from_function_response() required by the spec, and ADK would add these parts without any conversions to the contents - it would simplify agents that need to process binary files.
Currently, it's very non-intuitive to submit non-text Parts to the model, unless the user can construct the request to the model with one of those parts attached. For example, an agent that lists a set of images and let's the user indicate an image to be processed by an LLM currently requires a combination of a special tool to set the image reference in the state and a before-model-callback to check the state and add the Part.from_bytes() to the llm_request.contents and clear the state.
If a FunctionTool could return a list of parts, including the Part.from_function_response() required by the spec, and ADK would add these parts without any conversions to the contents - it would simplify agents that need to process binary files.