The advent of large language models (LLMs) in the field of AI has brought about a competitive edge for enterprises that have early access to these models. However, many organizations are left waiting due to regional availability constraints, resource challenges, bias, and language barriers. In response to this issue, Snowflake has announced the general availability of cross-region inference, providing developers with the ability to process requests on Cortex AI in a different region where models may be more readily available.
By enabling cross-region inference, developers can seamlessly integrate with the LLM of their choice, regardless of regional availability. This feature allows for the integration of new models as soon as they become accessible. Parameters for data traversal must be set to allow for cross-region processing, with the option to specify regions for inference. If both regions operate on Amazon Web Services (AWS), data will be securely encrypted as it traverses the global network. In the case of different cloud providers, traffic will be encrypted through mutual transport layer security (MTLS) over the public internet.
In order to execute inference and generate responses within the secure Snowflake perimeter, users must configure where inference will process by setting an account-level parameter. Cortex AI will automatically select a region for processing if the requested LLM is not available in the source region. For example, setting a parameter to “AWS_US” will allow inference to process in either the U.S. east or west regions. Currently, target regions can only be configured in AWS, meaning that requests will still process in AWS even if cross-region is enabled in Azure or Google Cloud.
A practical scenario provided by Arun Agarwal involves using Snowflake Arctic to summarize a paragraph. If the source region is AWS U.S. east and Arctic is not available there, Cortex will route the request to AWS U.S. west 2 for processing. The response will then be sent back to the source region, all accomplished with a single line of code. Users are charged credits based on the usage of the LLM in the source region, rather than the cross-region processing.
The introduction of cross-region inference in AI development presents a solution to the challenges posed by regional availability of large language models. By allowing developers to seamlessly integrate with LLMs regardless of location, organizations can accelerate their innovation processes and stay ahead of the competition. Snowflake’s initiative to provide cross-region inference in a secure and private manner opens up new possibilities for AI development on a global scale.