langchain_community.llms.sagemaker_endpoint.LineIterator¶
class langchain_community.llms.sagemaker_endpoint.LineIterator(stream: Any)[source]¶
A helper class for parsing the byte stream input.
The output of the model will be in the following format:
b’{“outputs”: [” a”]}
‘b’{“outputs”: [” challenging”]}
‘b’{“outputs”: [” problem”]}
‘…
While usually each PayloadPart event from the event stream will
contain a byte array with a full json, this is not guaranteed
and some of the json objects may be split acrossPayloadPart events.
For example:
{‘PayloadPart’: {‘Bytes’: b’{“outputs”: ‘}}
{‘PayloadPart’: {‘Bytes’: b’[” problem”]}
‘}}
This class accounts for this by concatenating bytes written via the ‘write’ function
and then exposing a method which will return lines (ending with a ‘
‘ character)within the buffer via the ‘scan_lines’ function.
It maintains the position of the last read position to ensure
that previous bytes are not exposed again.
For more details see:
https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/
Methods
__init__(stream)
__init__(stream: Any) → None[source]¶