Others aimed to describe the concept of cloud computing using a very broad, all-encompassing application of the virtual computing platform. The phenomenal growth of the Internet throughout the 1990s caused a huge reduction in the number of free IP addresses available under IPv4.
Principles of Parallel and Distributed Computing
- Parallel vs distributed computing
- Elements of parallel computing
- Elements of distributed computing
- Models for inter process communication
- Models for message-based communication
On the other hand, Event Systems architectural style where system components are connected and loosely coupled. In this case it is the publisher's responsibility to notify all subscribers.
Technologies for distributed computing
- On-demand Provisioning
- Universal Access
- Enhanced Reliability
- Multitenancy
- Resource pooling
- Rapid elasticity and Scalability
- Easy Maintenance
- High Availability
- Security
These demands are then automatically met by a cloud provider's service and users are charged only for their usage, i.e. the time they were in possession of the resources. The possibilities of the Cloud can be adapted and significantly expanded depending on use.
Web Services
This makes using web services seamless and straightforward with respect to technologies like CORBA that require much more integration effort. Therefore, lightweight alternatives to the SOAP/XML pair have been proposed to support web services.
REST and Systems of Systems
Web Service Description Language (WSDL) is an XML-based language for describing Web services. It is used to define the interface of a web service in terms of methods to be called and types and structures of the required parameters and return values.
Publish-Subscribe Model
Transferring data in an AJAX context is often preferred because, compared to XML, it is a lighter notation and therefore allows the same amount of information to be transferred in a more concise form.
Basics of Virtualization
Characteristics of virtualized environments
The guest program performs its activity by interacting with an abstraction layer, which provides access to the underlying resources. It becomes easier to control the performance of the guest by fine-tuning the properties of the resources exposed by the virtual environment.
Types of Virtualization
- Hardware level virtualization
- Hardware virtualization techniques
- Hardware assisted virtualization
- Full virtualization
- Paravirtualization
- Partial virtualization
- Operating system level virtualization
- Programming language-level virtualization
- Application level virtualization
- Other types of virtualization
- Network virtualization
- Desktop virtualization
- Application server virtualization
To make this possible, virtual machine drivers are required to provide a full emulation of the entire underlying hardware. Since they change the state of the resources exposed by the host, they must be contained in the virtual machine manager.
Implementation Levels of Virtualization
- Levels of virtualization implementation
- Instruction set architecture level
- Hardware abstraction level
- Operating system level
- Library support level
- User application level
Similarly, Wine provides library support for virtualizing x86 processors to run Windows applications on UNIX hosts. It uses the high performance of GPUs to run computationally intensive applications on host operating systems. Any program written in HLL and compiled for this VM will be able to run on it.
Virtualization Structures, Tools and Mechanisms
Xen architecture
The Xen hypervisor implements all the mechanisms, leaving the policy to be handled by Domain 0. The guest operating system, which has control capability, is called Domain 0, and the other is called Domain U. Therefore, one of the responsibilities of Domain 0 is to allocate and map hardware resources for the guest domains (the Domain U domains).
Binary translation with full virtualization
Its management VM is called Domain 0, which has the privilege to manage other VMs implemented on the same host. With full virtualization, non-critical instructions are executed directly on hardware, while critical instructions are detected and replaced by traps in the VMM that are emulated by software. An alternative VM architecture is to install a virtualization layer on the host OS.
Paravirtualization with compiler support
The guest OS kernel is modified to replace privileged and sensitive instructions with hyper-calls to the hypervisor or VMM. The guest OS running in a guest domain may be running in Ring 1 instead of Ring 0. This means that the guest OS may not be able to execute some privileged and sensitive instructions.
Virtualization of CPU, Memory and I/O Devices
- Hardware support for virtualization
- CPU virtualization
- Hardware-assisted CPU virtualization
- Memory virtualization
- I/O virtualization
- Virtualization in multi-core processors
- Physical versus virtual processor cores
- Virtual hierarchy
The VMM is responsible for mapping the physical guest memory to the actual machine memory. When the guest operating system changes the virtual memory to a physical memory map, the VMM updates the shadow page tables to enable direct lookups. The I/O access requests from the guest operating system are captured in the VMM that interacts with the I/O devices.
Virtualization Support and Disaster Recovery
Hardware Virtualization
Since the computing resources are shared by many users, a method is needed to maximize the user's privileges and still keep them securely separated. Users have full access to their own VMs, which are completely separate from other users' VMs. GAE provides limited application-level virtualization for users to build applications based only on the services created by Google.
Virtualization for IaaS
VM Cloning for Disaster Recovery
Process-level techniques are implemented on top of an existing operating system, which has full control over the hardware. This infrastructure layer serves as the foundation for building the platform layer in the cloud to support PaaS services. The platform layer is for general purpose and repeated use of the collection of software resources.
NIST Cloud Computing Reference Architecture
Cloud broker: An entity that manages the performance and delivery of cloud services and negotiates the relationship between cloud provider and consumer. Cloud carrier: An intermediary that provides connectivity and transport of cloud services from cloud providers to consumers. Cloud carrier is an intermediary that provides connectivity and transportation of cloud services between cloud consumer and cloud provider.
Cloud Deployment Model
- Public Cloud
- Benefits of choosing a Public Cloud
- Disadvantages of choosing a Public Cloud
- Private Cloud
- Benefits of choosing a Private Cloud
- Disadvantages of choosing a Private Cloud
- Community Cloud
- Benefits of Choosing a Community Cloud
- Disadvantages of Choosing a Community Cloud
- Hybrid Cloud
One of the main benefits that comes from using public cloud services is the almost unlimited scalability. At the top of the list is the fact that the security of data held within a public cloud is a cause for concern. A private cloud gives a single cloud consumer organization exclusive access and use of infrastructure and computing resources.
Cloud Service Model
Virtual machines are also the atomic components that are implemented and priced according to the virtual hardware's specific features: memory, number of processors, and disk storage. From the service provider's perspective, IaaS/HaaS allows better utilization of the IT infrastructure and provides a more secure environment in which third-party applications are executed. At the same time, users can benefit from the full customization that virtualization offers to deploy their infrastructure in the cloud.
PaaS
Other implementations of the PaaS model provide a complete object model for representing an application and offer a programming language-based approach. In the second case, which this book calls Pure PaaS, the middleware is the core value of the offering. Runtime Framework: This framework represents the software stack of the PaaS model and the most intuitive aspect that comes to people's minds when they refer to PaaS solutions.
SaaS
Among these, the most popular is Microsoft Windows Azure, which provides a comprehensive framework for building service-oriented cloud applications on top of .NET technology, hosted in Microsoft data centers. On the customer's side, such costs constitute a minimal portion of the usage fee paid for the software. SalesForce.com is built on top of the Force.com platform, which provides a complete environment for building applications.
Architectural Design Challenges
- Challenge 1: Service Availability and Data Lock-in Problem
- Challenge 2: Data Privacy and Security Concerns
- Challenge 3: Unpredictable Performance and Bottlenecks
- Challenge 4: Distributed Storage and Widespread Software Bugs
- Challenge 5: Cloud Scalability, Interoperability, and Standardization
- Challenge 6: Software Licensing and Reputation Sharing
In a cloud environment, recent attacks may be due to hypervisor malware, guest hops and hacking, or VM rootkits. Multiple VMs can share CPUs and main memory in cloud computing, but I/O sharing is problematic. The opportunity here is to scale up and down quickly in response to load variation, to save money but without violating SLAs.
Cloud Storage
- Storage-as-a-Service
- Advantages of Cloud Storage
- Cloud Storage Providers
- Runtime Support Services
Storing data in the cloud is beneficial as it allows the user to protect data in the event of a disaster. Platform as a Service (PaaS) solutions provide a development and deployment platform for running applications in the cloud. The end user can access data stored in the cloud via an Internet connection.
Resource Provisioning
A distributed scheduler for a cloud application has special features that can support cloud applications, such as scheduling programs written in the MapReduce style. As a result, there is no upfront investment in servers or software licensing on the customer's side. Customer data is stored in a cloud owned by the vendor or in a publicly hosted cloud that supports PaaS and IaaS.
Resource Provisioning Methods
Demand-Driven Resource Provisioning
This method adds or removes compute instances based on the current level of usage of the allocated resources. In general, when a resource has exceeded a threshold for a certain amount of time, the scheme increases that resource based on demand. When a resource is below a threshold for a certain amount of time, that resource can be scaled back accordingly.
Event-Driven Resource Provisioning
Popularity-Driven Resource Provisioning
Global Exchange of Cloud Resources
Cloud providers will be able to dynamically expand or change their provisioning capacity based on unexpected increases in workload demands by leasing available compute and storage capacity from other cloud service providers. Act as part of a market-driven resource leasing federation where application service providers like Salesforce.com host their services based on negotiated SLAs driven by competitive market pricing. It supports the trading of cloud services based on competitive economic models such as commodity markets and auctions.
Cloud Security Challenges
In the cloud computing environment, the company subscribes to cloud computing resources, and the responsibility for patching lies with the subscriber and not the cloud computing providers. Data is fluid in cloud computing and can reside on on-premises physical servers, on-premises virtual machines, or off-site virtual machines running on cloud computing resources, and this will require some reconsideration from both auditors and practitioners. In the world of cloud computing, the cloud computing provider is responsible for the security and privacy of customer data.
Software-as-a-Service Security
To create trust zones in the cloud, the virtual machines must defend themselves and effectively move the perimeter to the virtual machine itself. Recovery is the way to find out what happens to data in the event of a disaster. To address the security issues mentioned above, SaaS providers will need to integrate and improve the security practices used by the managed service providers, and develop new ones as the cloud computing environment evolves.
Security Governance Managed
A charter for the security team is usually one of the first products from committee leaders. This charter should clearly define the roles and responsibilities of the security team and other groups involved in performing information security functions. A lack of governance and proper task management can also result in potential security risks being left unaddressed and opportunities to improve the business being missed because the security team is not focused on key security functions and activities that are critical for business.
Virtual Machine Security
By introducing this traditional line of defense into the virtual machine itself, the user can enable the secure transfer of critical applications and data to the cloud. To facilitate centralized management of the server's firewall policy, the security software loaded on the virtual machine should include a two-way stateful firewall that allows for virtual machine isolation and location detection, allowing for tighter policy and the flexibility to move the virtual machine from premises to cloud resources. This approach to virtual machine security, which ties the machine back to the mothership, has some advantages in that the security software can be placed in a single software agent that provides consistent control and management across the cloud while seamlessly integrating back into the existing the security of infrastructure investments that provide economies of scale, deployment and cost savings for both the service provider and the enterprise.
Security Standards
Open Authentication (OAuth)
Google's DeWitt Clinton heard about the OAuth project and expressed interest in supporting the effort. OAuth allows users to grant access to their information, which is shared between the service provider and consumers, without sharing their entire identity. It is important to understand that security and privacy are not guaranteed by the protocol.
OpenID
After receiving such information, the user calls the read function to connect to the nearest DataNode containing the first block of the file. Since each file block must be replicated by a predefined factor, the data streamer first sends a request to the NameNode to get a list of suitable DataNodes to store replicas of the first block. The process continues until all assigned DataNodes receive a replica of the first block of the previous DataNode.
MapReduce
The TaskTracker controls the execution of the map and/or reduce tasks on a single compute node in the cluster. Task Assignment The JobTracker creates one map task for each computed input partitioned by the user node and assigns the map tasks to execution slots of TaskTrackers. The JobTracker considers the location of the data when assigning the map tasks to the TaskTrackers.
Virtual Box
So, after the sorting and shuffling phase, each reducer will have a unique key and a list of values corresponding to that very key. As shown in the figure, reducer gets a list of values that are [1,1] for the key Bear. Finally, all the output key/value pairs are then collected and written to the output file.
