Well designed SAP cloud computing for SAP and S/4HANA workloads on hybrid multi-cloud environments implements architecture, integration, network and security best practices.
Modern cloud computing concepts, with serverless, containerized and event based architectures, are cloud provider agnostic and enable implementations of resilient, scalable and high available applications on cloud platforms like Microsoft Azure, Amazon AWS or SAP Business Technology Platform (SAP BTP).
Serverless architectures facilitate cloud computing with integrated managed cloud services and costs calculated by invocations or runtime usage. Underlying infrastructure with servers, databases, network and storage gets completely managed by cloud providers.
Serverless functions offer Bring Your Own Language (BYOL) concepts to implement business logic with different programming languages. Azure Functions or AWS Lambda support implementations with programming languages like JavaScript, Java, Python, C# and others. Kyma functions of the SAP Business Technology Platform Extension Suite can be currently (02/2022) realized with Python or Node.js programming languages.
Container virtualize software layers to run applications in virtualized environments. They offer lighter weight virtualization and shorter startup times than virtual machines which also handle hardware virtualization. Kubernetes is mostly used to orchestrate containers deployed in runtimes of container engines like Docker container.
Decoupled communication scenarios between decentralized cloud systems or services are typically based on APIs, asynchronous messaging or events. Events are lightweight notifications, used in publish-subscribe pattern to broadcast (fan out) data objects of interests with references like URLs or identifiers. In contrast to events, messages contain complete data objects to be processed by the receiver.
Cloud software provided over the internet depends on highly performant and available web applications, with the ability to handle traffic spikes with scalability and elasticity features, independent of client locations.
Distributed content server offer delivery networks (CDN) for static web caching to reduce the latency of web client request. The combination of Anycast routing, with single IP addresses shared by servers in multiple locations, and the usage of the Border Gateway Protocol (BGP) enables routing to closest available content servers. Exchanging routing and reachable information between autonomous network groups also enables automated rerouting in case of failures.
Azure DNS, Azure Traffic Manager and AWS Route53 services offer low-latency domain name resolving with anycast routing and temporary caching.
Load balancing can optimize the load distribution of web applications and improve the availability of websites with alternativ routes for unhealthy routing targets on different OSI communication levels. DNS load balancing can be implemented with Azure Traffic Manager or AWS Route 53.
The usage of the SAPUI5 CDN is restricted to SAP Cloud Products only. To overcome this limitations, SAPUI5 libraries can be hosted on Azure CDN or Amazon CloudFront.
This Microsoft Azure reference architecture describes best practices for highly scalable Azure App Service web applications.
Modern websites of SAP cloud applications typically consume RESTful OData APIs which can be hosted with the HTTP-based Azure App Service. Microsoft recommends creating the web application and the web API as separate App Service apps to enable independent scaling.
In this reference architecture, incoming HTTP requests are handled by the FrontDoor global level 7 application load balancer service to increase performance and security with routing options like rules or caching and integrated web application firewall (WAF) capabilities.
Azure Function apps run serverless background tasks decoupled by the Azure Queue Storage from the App Service web application. This design offers buffering of web application requests and enables independent application component scaling.
Caching strategies improve the performance of web requests and reduce at the same time application or database load. The Azure Cache for Redis is a memory data store which reduces the database load with a data layer cache.
Azure CDN and Azure FrontDoor are two content delivery network (CDN) options with unique edge caching capabilities. Azure CDN is optimized to deliver file content like images, videos or documents from storage. Azure FrontDoor can be used to offer sites, services or APIs. Both CDN options support high availability by forwarding traffic to alternative servers in case of failures.
This well architected AWS reference web application demonstrates implementation options for cloud qualities like scalability, performance and high availability. Decoupled web and application components are deployed on own servers into two separated availability zones. Load balancers distribute incoming traffic, based on the availability of the web application components, into these availability zones.
Scalability is enabled with the deployment of web application components on EC2 compute instances in auto scaling groups.
Caching improves the performance of client requests with Amazon CloudFront for web content near the edge and with Amazon Redis on the database layer as in-memory store for query results. Scalability, availability and performance of static web content access to websites, images or videos is realized with Amazon S3 as high volume object storage.
Azure Function apps run code serverless, triggered by events like HTTP requests, timer or services with automatically scaled managed infrastructures. Assigned hosting plans need to be aligned with implementation requirements e.g. to minimize management overhead with consumption plans or to improve cold start behavior using pre-warmed instances with premium plans. Azure Durable Functions allow implementing stateful behavior to orchestrate workflows.
Azure Functions hosting models support different levels of network isolation. Consumption plans offer only minimal network isolation options in contrast to other plans which support e.g. inbound private endpoints, virtual network integration or outbound IP restrictions. Public read-only access can be implemented with authorization level Anonymous combined with a Get Only request method. Further authorization levels are function with a specific API key or admin with the function master key.
Azure Container Instances are the fastest way to run
isolated containers without orchestration for simple applications, task
automation and build jobs. Container groups simplify the development and
single file deployment of multi-container architectures, with resource
manager templates to include further resources or YAML files. They
enable container scheduled together on shared resources, similar to
Kubernetes pods.
Container groups can divide one functional task to
multiple containers like main application with supporting role
containers (e.g. to serve backend, web, monitoring, logging or pipeline
parts).
Azure Kubernetes Service (AKS) is fully managed and offers a control plane which simplifies the management, deployment and health monitoring of containerized services with autoscaling and coordinated upgrades. AKS helps to modernize and deploy existing applications packaged as containers e.g. ASP pages with third-party components (DLLs) registered in the Windows registry.
The fully managed Azure App Services PaaS supports multiple programming languages to implement containerized cloud optimized web or mobile apps and RESTful APIs. VNet Integration enables apps to access private virtual network resources or on-premises resources by integrating with VNets that have site-to-site connections.
App Service plans define a set of compute resources with different pricing tier related features to run code serverless without explicitly provisioned and managed infrastructure. Production workloads can run on standard or premium plans with features like autoscaling. Isolated plans run dedicated VMs with maximum scale-out capabilities for mission critical workloads and are also used in Azure Service Environments (ASE).
Basic, Standard, Premium, Isolated and ASE v2 SKUs Support hosting of Azure Functions. ASE v3, based on isolated v2 plans, supports hosting of Logic Apps (Standard) and Functions.
Azure offers different options to implement long running jobs e.g. image processing like Azure Functions which guarantee 60 min with the premium plan type. Web Jobs are available for longer running jobs with App Service plans of Web apps without additional costs and Always on setting to run reliable. WebJobs can be used with any program or script that runs in the App Service sandbox.
Logic apps are an option to run scheduled background image processing with costs for executed tasks. Azure Durable Functions are an option if the long running job can be splitted into smaller pieces.
Azure Batch can be used to create render farms with parallel task execution enabled on compute nodes. Low-priority virtual machines (VMs) reduce costs of batch workloads with the tradeoff that subscribed VMs may not always be available to be allocated.
Azure Storage Accounts can contain all data objects with different account types like general-purpose V2 for all storage services (blob, file, table, queue) and premium storage (blob, file). Replication and data isolation requirements have to be considered with regard to storage costs which get calculated according to region, account type, access tier, capacity, redundancy, transactions and data egress.
Data is always stored redundant with different options. Local redundancy (LRS) protects against e.g. server, rack failures, zone redundant storage (ZRS) ensures availability in case of zone disaster, geo-redundancy (GRS / GZRS) protects against regional failures with the option to add read access (RA-GRS / RA-GZRS) to the secondary region.
Azure Queue Storage can store large numbers of messages to create backlogs of work for asynchronous processing which enables cloud services to communicate transaction Information asynchronously.
Azure Files Shared storage service offers Server Message Block (SMB on Windows) or Network File Share (NFS on Linux) file-sharing protocols. Azure Files can directly be mounted, cached with Azure File Sync and used from containers with built-in CSI drivers.
Azure Table table API uses a key/attribute store for schemaless designs. Azure Disks can be assigned to virtual machines as data storage.
Shared Access Signatures (SAS) provide secured access to specific storage account resources with defined permissions and time validation to users, services or accounts.
Azure Blob object storage is optimized for massive amount of unstructured data (e.g. images, files, video, audio, log, backups or data) for further analysis and support for Azure Data Lake Storage Gen2. Objects can be accessed via HTTP requests with client libraries (e.g. Java, .NET, Node.js, Python).
Storage types and access options can be set according to individual requirements like premium storage type for IO intensive workloads that perform many small transactions.
Access tiers classify storage according to the access frequency. The default access tier Hot is optimized for frequent reads and writes of objects with lowest access costs. The Cool access tier is optimized for large amounts of infrequently, rarely accessed data to be stored at least 30 days with immediate availability. Data in the Archive tier must remain for at least 180 days for unlikely access of file-based database backups.
Immutable storage supports two types of policies, time-based retention objects cannot be overwritten but deleted after the retention period has expired. Legal hold protects objects against modifications and deletions until the policy is explicitly cleared.
Azure Cosmos DB is a globally distributed, multi-model database with elastic scalability and single-digit-millisecond data access via SQL or NoSQL APIs. Cosmos DB offers the availability SLA of 99.999% for data distributed over multiple regions. The Cosmos DB table API handles data with less latency and higher throughput compared to table storage. MongoDB can be migrated to Cosmos DB using APIs and the mongorestore tool.
Azure SQL Server can run on VMs with OS level access and offers Windows authentication.
Azure offers managed SQL Databases and SQL Managed Instances with the option to manage multiple databases. Elastic SQL Database or Managed Instance pools are scalable cost-effective solution on single servers with shared compute and storage resources.
Managed Instances are SQL Server versions with limited functionality to ensure high-availability as PaaS service offered with DTU and vCore based purchasing models. The database transaction unit (DTU) model is only available for SQL Databases with service tier Basic, Standard or Premium. DTUs offer a blend of compute, storage and IO read-writes and allow changes with minimal downtimes.
Virtual Core (vCore) based purchasing models offer the option to choose physical hardware characteristics, like number of cores, memory and storage size, independently and enable read scale-out. This model is available with Premium and Business Critical service tiers for SQL Databases and Managed Instances.
Azure SQL Database can place different replicas of the Business Critical database to different availability zones in the same region with availability SLA of 99,995%, RPO of 5 sec and 30 sec RTO. The Hyperscale service tier supports up to 100 TB of database size with rapid scale-out capabilities. Auto-scaling is enabled with the serverless compute tier which is available in the General Purpose service tier and currently (02/2023) in preview in the Hyperscale service tier.
Some features of the Azure SQL Managed Instances are stored procedure implementations with the Common Language Runtime (CLR), max instance reserved General Purpose storage of 8TB and service broker in-database message queueing. Standard and Premium pricing tiers support retaining daily backups for up to 35 days and monthly backups for up to 120 months.
Azure Data Migration Assistant (DMA) can be used to migrate SQL Server Integration Services (SSIS) packages with data integration content (e.g. control or data flow, connections) to target SQL databases or Managed Instances.
Azure virtual machines are computing resources which offer options to manage availability zones to reach 99.99% availability and scale sets with load balancing.
Premium storage disk types are designed for high performance speed and availability for data of operating systems, databases and logs. Host caching strategies can further improve the performance of virtual machines with policies None for log files and ReadOnly for workloads without write operations.
Network interface cards (NIC) enable the communication of virtual machines within the Azure network, to on-premise or the internet. Default NIC get created automatically with VNet In-/Outbound, Load Balancer Inbound and Internet Outbound allowed.
Network Security Groups (NSG) filter traffic with rules between Azure resources. NSG Traffic Analytics flow logs of VMs provides visibility about activities like e.g. internet access. Azure Service Maps allow to visualize different processes and their dependencies.
Azure virtual machines offer different BCDR Scenarios with local redundant storage (LRS) as lowest cost option for backups. Data center failures can be handled with scale sets as load balancer targets deployed across two availability zones.
The fastest Recovery Time Objective (RTO) for failover scenarios is provided by Azure Site Recovery within 5-10 min, compared to Azure Backup RTO with approximately 6 hours. Unmanaged disks with GRS offer RPO of approx. 15 min but without SLA.
Azure Monitor collects telemetry data of cloud and on-premises environments into a common data platform which is based on Azure Data Explorer with similar log data tables. Data from multiple tenants or subscriptions and different sources like applications, operating systems, custom sources or cloud resources like containers or virtual machines can be analyzed with aggregations or correlations to be visualized for reporting.
Metrics of numerical values, logged events, event traces across distributed systems and event changes enable monitoring functionality and security aspects of applications or infrastructure components on Azure. Azure Monitor can stream monitoring this data also to external resources via Event Hubs. Data retains available within interactive retention periods with default 90 days e.g. for application insights and max 730 days.
Azure Log Analytics is a tool in the Azure Portal to run queries on Azure monitor data. Log queries can be written with the Kusto query language (KQL) e.g. to correlate the usage and performance data collected by Application Insights. Log Analytics is built on top of Azure Data Explorer and works with log data tables separated in workspace.
Application Insights provides Application Performance Monitoring to track user interface interactions, requests to specific code lines or to collect and analyze application health, performance and usage data. Application maps can identify performance bottlenecks between distributed components.
Azure Monitor Activity log provides insights into subscription-level events primarily for activities that occur in Azure Resource Manager. Activity Log Insights offer reports on ARM deployments within a subscription based on the logs stored with a retention period of 90 days. Azure Resource Health informs about problems and unavailability of service instances (resources) because of Azure service problems which allows to check SLA violations.
Azure AD Connect Health monitors on-premise identity infrastructures with AD Federation Server (AD FS), Web Application Proxy (WAP) server and Azure AD Connect
Azure Network Watcher runs IP flow verify to analyze the network traffic e.g. to determine whether packets are being allowed or denied by VMs
Action Groups define actions and notifications to be executed for triggered alerts of Azure Monitor or Service Health. Alerts get defined with target, signal and logic to notify about metrics, log search, activity log or platform health.
Azure SQL Insights uses Dynamic Management Views to monitor health, diagnose problems or tune performance with wait statistics and memory Information of Azure SQL family databases. Collecting Telegraf agents on separated VMs gather telemetry data to be stored in Azure Monitor log tables.
Azure SQL Database Diagnostics offers telemetry data streaming to SQL Analytics, Azure Storage or Event Hub (for Power BI real-time reporting).
Performance Analysis of long running queries can be performed with Intelligent Insights (SQL Database, Managed Insights) or Query Performance Insights (SQL Databases).
Azure VM insights monitors performance, dependencies and connections of components or services on virtual machines or servers. Enabling VM insights triggers the installation of the Dependency and either Azure Monitor or Log Analytics agents. Dependency agents collect discovered data about processes running on virtual machines and external process dependencies used by the VM insights map feature.
Log Analytics Agents collect monitoring data from guest operating systems and workloads of virtual machines in Azure, other clouds or on-premise. Installed Microsoft Monitoring Agent on VMs allows to centrally monitor all warning events in the system logs and to send data to Log Analytics workspaces.
Diagnostics VM Extensions collect monitoring data only from Azure virtual machines, such as performance counters or event logs and send data to an Azure Storage account, Event hub or Azure Monitor Metrics (for Windows).
Azure AD centrally manages all apps within a tenant and assigns an unique Application ID in the app registration:
The Azure Enterprise Apps SAP Cloud Platform can be used to access Business Technology Platform services like SAP Mobile Services. Enterprise app SAP Identity Authentication Service is used to establish a trusted connection to Identify Authentication Service (IAS) service as proxy for Azure AD.
The managed Azure App Configuration service offers capabilities to manage application settings or features centrally. App configurations get stored with key-value pairs separated from coding according to the twelve-factor apps architectural pattern.
App Configuration and Azure Key Vault are complementary services often used side by side in application deployments. Virtual machines can retrieve imported security keys, generated with on-premise HSM, from Key Vaults with service principals to encrypt disks.
Managed identities provide identities for applications to authenticate connections to resources supported by Azure AD like Azure Key Vault, Azure SQL or Cosmos DB.
Fully managed AWS services enable build and run serverless applications.
| Area / Services | Short Description |
|---|---|
| Compute | |
 |
AWS
Lambda Processing: event-driven, isolated and stateless serverless computing platform. AWS Lambda functions execute code triggered by user events, data or system state changes. Lambda can also be triggered directly by AWS services or custom applications. JSON is used as function input and output format. Use Cases: e.g. changes of S3 buckets, updates of DynamoDB or streaming data processing with AWS Kinesis Networking: additional costs for HTTP integration with API Gateway or Elastic Load Balancer have to be considered. Inbound network connections are blocked and for outbound only TCP/UDP is allowed. Features:
|
 |
Cloud
Front is a Content Delivery Network (CDN) service. Distributed
caches accelerate access to website content and reduce infrastructure
load of AWS services such as Amazon S3 or EC2. CloudFront enables HTTPS
access to AWS S3 static websites. CDN is recommended to reduce costs and latency of web applications. Lambda Edge is a CloudFront feature to run code closer to users. |
 |
Application Load Balancer supports path and host based routing on the seventh OSI protocol layer. Targets lambda functions. |
| Amazon ECS | Elastic Container Service is a proprietary and simple to manage service. Exclusively for and deeply integrated with the AWS cloud platform |
| Amazon EKS | managed Kubernetes service with Kubernetes control-plane and flexible run options |
| AWS Fargate | serverless container management service. Enables container orchestration without the need to manage the infrastructure like servers or VMs |
| Workflows | |
 |
Step Functions coordinate multiple AWS services into workflows |
| Storage | |
 |
AWS S3
|
 |
AWS EFS fully managed, scalable NFS file system with POSIX compliant file level permissions |
| Data Store | |
 |
DynamoDB fully-managed NoSQL database with key-value and document store. Serverless, scalable, single-digit-millisecond performance and in-memory caching |
 |
Aurora Serverless auto-scaling configuration of relational Aurora database (MySQL and PostgreSQL compatible) |
 |
AWS Kinesis real time aggregation and analysis of data streams |
| Messaging | |
 |
SNS Messaging highly available, fully managed publish-subscribe messaging service |
 |
SQS Queue with two types of message queues, standard with for maximum throughput and FIFO |
| Integration | |
 |
API Gateway is an API Management platform for developers working with REST, HTTP and WebSocket APIs |
Amazon EC2 offers a wide range of options as Infrastructure as a Service (IaaS), to move existing applications to the cloud. These existing applications can be easily deployed with Elastic Beanstalk and orchestrated in containers with Elastic Container Services.
| AWS Service | Short Description |
|---|---|
 |
CloudWatch collects monitoring and operational data for AWS cloud resources and the applications in the form of logs, metrics, and events. Provides a unified view of AWS resources, applications and services |
| AWS CloudTrail | log actions to enable governance, compliance, operational and risk auditing |
| AWS Config | enables to assess, audit and evaluate the configurations of your AWS resources. Can create OpsItems for non-compliant resources |