Skip to content

'AsyncAnthropicBedrock' object has no attribute 'messages' #380

Description

@sk-g

Trying to invoke Sonnet following the docs https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html#api-inference-examples-claude-messages-code-examples.

However, I am trying to use this in async context through bedrock.

from anthropic_bedrock import AsyncAnthropicBedrock

seconds_to_sleep_each_loop=60
remaining_seconds_to_pause=60
def start_a_client(timeout=40.0, max_retries=0):
    client = AsyncAnthropicBedrock(
                    timeout=timeout,
                    max_retries=max_retries,
                )
    return client

async def run_api(
        client,
        data,
        out_path,
        task="generate_random",
        max_tokens_to_sample=1024,
        debug=True,
        model="anthropic.claude-v2:1"
    ):
    messages = [
        {
            "role": "user",
            "content": data,
        }
    ]   
    body=json.dumps(
        {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens_to_sample,
            "system": "",
            "messages": messages
        }
    )

    try:
        completion = await client.messages.create(
            model=model,
            max_tokens_to_sample=max_tokens_to_sample,
            messages=messages,
        )
       # .... rest of the code
    except Exception as e:
        print(e)
        return None

async def query_in_batch(client, data_list, out_path, max_tokens_to_sample=1024, debug=False):
    client = start_a_client()
    for line in data_list:
        task = "generate_random"
        asyncio.create_task(
             run_api(
                    client,
                    line, out_path, task=task,
                    max_tokens_to_sample=max_tokens_to_sample
            )
        )
        await asyncio.sleep(seconds_to_sleep_each_loop)
    await asyncio.sleep(remaining_seconds_to_pause)

yields the following error on invocation within async calls

'AsyncAnthropicBedrock' object has no attribute 'messages'
  1. Is messages api not enabled in anthropic_bedrock?
  2. Is there documentation for async calls to sonnet?

Appreciate the help

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions