Memory Hierarchy Design and Characteristics
Memory Hierarchy Overview
Our computer systems consist of various kinds of computer memory in modern computer systems, like register, cache memory, main memory, flash memory, magnetic disks, magnetic tapes, etc, each of these components has different performance rates and specific usages.
Memory Hierarchy is the meaningful arrangement and visualization of these various memory devices concerning their performance, access time, and cost per bit; the computer memory hierarchy is a concept in computer architecture that proves helpful while designing a new system to balance its overall performance-to-cost ratio.
What is Memory Hierarchy?
When a Computer System is designed, a large number of different devices are used in it, like processors, different memory devices, motherboards, etc., and these parts are very expensive, which means we are not free to use them as many numbers of times as we want, meaning we are bound to use each particular part only a specific number of times in the design, to balance the overall cost.
Now, talking about the memory devices in a system, when a computer system is designed, a wide variety of memory devices are used, the most commonly used memory device that we all are familiar with is RAM, the Random Access Memory. But it is not the only memory that is used in our systems, we have registers, cache memory, flash memory, magnetic disks, magnetic tapes, etc. different kinds of memories as well.
So what is memory hierarchy?
The memory hierarchy is the memory organization of a particular system to balance speed, capacity, and cost efficiency. As a system has several layers of memory devices, all having different performance rates and usage, they vary greatly in size and access time as compared to one another.
The memory Hierarchy provides a meaningful arrangement of these various memory types to maximize the performance of a system and improve overall system performance.
Let’s understand each of them one by one
Memory Hierarchy Design
The above diagram shows a clear understanding of the memory hierarchy design, and you might wonder, why the shape is a pyramid?, it is to reflect the size (in bits) and access time of each of these memory layers, along with their different access speeds, as the most important difference that distinguishes these layers is the size in which they are available inside a system and the access time of theirs concerning the CPU (Central Processing Unit); together, these layers form the memory system that organizes memory resources efficiently.
Let’s understand each of these levels one by one
Registers
These are very small memory devices, generally holding around bytes of data nearly 32-bit to 64-bit, and are of various types, for example - general-purpose registers, and address registers that hold or reference a memory address. The registers are directly associated with the CPU which means they are the fastest memory available in a system, which also means they are the costliest of all, Due to this they are used in very small sizes and possess the fastest access time.
Cache Memory
Cache memories are also called SRAM (Static Random Access Memory, or static ram), they are measured in kilobytes or megabytes, and are used to store the frequently used segments of programs accessed by the processor. Cache memory stores frequently accessed data and instructions because many programs exhibit locality. This benefit comes from temporal locality, where the same data is reused soon, as opposed to spatial locality, where nearby data is likely to be accessed next. Registers and Cache memories are embedded on the CPU itself, and hence are the fastest and are collectively referred to as internal memory.
Transform Your Career
Choose from our industry-leading programs designed for career success
Modern Software and AI Engineering Program
Master full-stack development with AI integration
+1000 moreModern Data Science and ML with specialisation in AI
Advanced data science techniques with AI specialization
+1000 moreAdvanced AIML with Specialisation in Agentic AI
Deep dive into AIML with focus on Agentic systems
+1000 moreDevOps, Cloud & AI Platform Engineering
Build and manage AI-powered cloud infrastructure
+1000 moreAI Engineering Advanced Certification by IIT-Roorkee
Premier AI engineering certification from IIT-Roorkee
Main Memory (RAM)
This is the most commonly used memory, also called DRAM (dynamic ram, Dynamic Random Access Memory), the main memory directly communicates with the CPU and auxiliary memory devices through input and output, including data transfer between main memory and other devices. These are less expensive as compared to cache memory and are measured in Gigabytes.
All the above three memories ie, Registers, Cache Memory, and Main memory are volatile, with RAM used to store data and instructions currently needed by the processor, meaning all the data inside them will be erased if the power is cut off.
Flash Memory
Memories like Pendrives and Solid State Drives come in this category and are generally measured in Gigabytes.
Magnetic Disks
Most commonly available memories are slower than flash memories; storage like magnetic disks falls in this category, with traditional disks using spinning circular plates, offering larger storage capacity but slow access times, and are measured in Gigabytes or Terabytes.
Flash Memory and magnetic disk storage are collectively referred to as Secondary Storage.
Scaler Placement Report and Statistics
Scaler learners achieved 2.5x salary growth with average post-Scaler CTC reaching ₹23L.
Magnetic Tapes
Magnetic tapes are the largest in size provide the slowest access time, and are generally measured in Petabytes, they are mainly used as backup storage and are not directly connected to the CPU.
Magnetic tapes are referred to as tertiary storage.
Cost
Now if we talk about the cost, then you may think, that in Registers we are storing only some bytes of data, and in Magnetic Tapes we are storing petabytes of data (nearly 1015 bytes of data) and hence Registers should cost less and the Magnetic tapes should cost more, but it is the opposite which is true, Registers are costliest among all of them, and Magnetic tapes, are the cheapest if we talk about the price to memory (in bit) ratio.
The reason for this cost variation is their access time, meaning if we try to access data present in registers, it is going to be the fastest, After comes the cache memory, then the main memory, then flash drives, then magnetic disks, and finally, the slowest accessing time will be in magnetic tapes.
Access Time
Let's have a look at the below table, to have a glimpse of access times for each of the memory levels.
| Memory | Type | Access Time (Approx) |
|---|---|---|
| Registers | Internal Memory | 1ns |
| Cache Memory (SRAM) | Internal Memory | 10ns |
| Main Memory (DRAM) | Primary Memory | 100ns |
| Flash Drives | Secondary Storage | 1μs |
| Magnetic Disks | Secondary Storage | 10ms |
| Magnetic Tapes | Tertiary Storage | 100ms |
Frequency
If we talk about the frequency ie, which memory is used most frequently by the CPU, then they are registers, as they are directly embedded onto the CPU, and we know that even to do the smallest of the tasks, the CPU accesses the registers, so they are the most used memory in any system. And the least used memory device is Magnetic tapes, as they are generally used for keeping the backups, and we know that on a regular basis, backups are not accessed, so they are the least used memory in a computer system.
Characteristics of Memory Hierarchy Design
Performance
Long ago, the designing of computer systems was not implemented using memory hierarchy, due to which the speed difference among different memory components and CPU registers keeps getting enhanced gradually and which resulted in lower performance of systems, so to improve that, memory hierarchy designs were introduced, which improved overall system performance and resulted in better performance in modern computers.
Turn Learning into Career Growth
Access Time
Access time is the time interval between the availability of the data and the subsequent read/write requests, and it depends on how quickly memory access can be completed.
Access time increases if we travel from top to bottom in the memory hierarchy design.
Capacity
As we travel from top to bottom, the capacity increases, it simply means the volume of data the memory can store.
Cost Per Bit
The ratio of cost per bit increases as we go from bottom to top, meaning the registers are the costliest memory and secondary and tertiary storage are the cheapest.
Types of Memory Hierarchy in Operating System
The Memory Hierarchy is divided into two primary categories, a common classification in computer science discussions of memory hierarchy.
Internal Memory
Internal memory is also referred to as primary memory, in a computer system, internal memory is memory that stores the data that needs to be accessed frequently and quickly, this type of memory is directly reachable by the system and other processes through input/output modules. It consists of
-
Registers
-
Cache Memories
-
RAM
-
ROM
External Memory
Also known as Secondary Memory, these are peripheral storage devices used to store huge amounts of data. They are non-volatile memories, meaning they will be storing the data even if the power gets off. They consist of
-
Magnetic Disks
-
An optical disk
-
Magnetic Tapes
Advantages of Memory Hierarchy
There are several advantages of using a memory hierarchy in computer systems, including:
-
Faster Access: By having multiple levels of the memory hierarchy, computer systems can provide faster access to frequently accessed data by exploiting different access speeds across levels. The fastest memory such as cache memory is used to store the most frequently used data, which can be accessed much faster than other types of memory.
-
Cost efficiency: The use of a memory hierarchy makes the computer system more cost-efficient, as implementing high-speed memory such as cache memory is more expensive than other types of memory like main memory or secondary storage. With the use of a memory hierarchy, the most expensive memory can be used where it is needed the most, without needing to use it everywhere.
-
Efficient use of resources: Memory hierarchy allows efficient use of memory resources by storing frequently accessed data in fast memory and less frequently accessed data in slower memory. This ensures that the computer system does not waste resources by using high-speed memory for data that is rarely accessed.
-
Increases the processing speed: The use of a memory hierarchy allows computer systems to perform operations much faster. By using the fastest memory for frequently used data, the CPU can access the data quickly and complete the operations faster, increasing the processing speed of the system.
-
Increased capacity: Memory hierarchy increases the overall capacity of the computer system by allowing secondary storage to hold data stored in larger volumes while high-speed memory keeps the most active items. This allows the system to handle large amounts of data without slowing down its performance.
Overall, using a memory hierarchy in computer systems helps improve the system’s overall performance and efficiency, making it faster, cost-effective, and efficient in its use of resources.
Disadvantages of Memory Hierarchy
The memory hierarchy design provides a solution to optimize the overall performance of a system while balancing the cost. However, it also has some disadvantages:
-
Complexity: It adds complexity to the system as there are multiple layers of memory, and the system needs to manage them efficiently.
-
Access time: Although the memory hierarchy design aims to reduce the access time to the data, it still adds some latency in the system. The data may need to move from one layer to another, which can delay transfer and increase the access time.
-
Cost: While memory hierarchy design aims to balance the overall cost, it still adds to the cost of the system. The higher the number of layers, the more costly the system becomes.
-
Limited capacity: Each layer of the memory hierarchy has limited capacity, which means that the system cannot store an unlimited amount of data, even with virtual memory. This limitation can sometimes lead to performance issues.
-
Management overhead: The system needs to manage the data movement between the different layers of memory, which adds overhead to the management process.
Despite these disadvantages, memory hierarchy design is still widely used as it provides an efficient way to manage memory and optimize system performance.
Conclusion
-
In a computer system various types of memories are used, in order to balance the cost-to-performance ratio, Memory Hierarchy is the arrangement of these various memory types.
-
Look at the below table.
| Memory Types | Access Time (Approx) | Cost per bit |
|---|---|---|
| Registers | 1ns | Highest cost |
| Cache Memory (SRAM) | 10ns | Lesser than Registers |
| Main Memory (DRAM) | 100ns | Lesser than Cache Memory |
| Flash Drives | 1μs | Lesser than Main Memory |
| Magnetic Disks | 10ms | Lesser than Flash Drives |
| Magnetic Tapes | 100ms | Lesser than Magnetic Disks (Lowest cost) |
-
Memory Hierarchy is designed based on the performance of a specific memory type, its access time, its capacity to store data in it, and its cost per bit.
-
Memory Hierarchy is classified into two types
-
Internal Memory (Register, Cache Memory, RAM, ROM)
-
External Memory (Magnetic Disks, Optical Disks, Magnetic Tapes)
-
-
Memory Hierarchy provides various advantages like visualizing a better cost-to-performance ratio, designing systems that can be properly planned in a better and more managed way, etc.
Google
Amazon
Microsoft
Flipkart
Adobe1200+ more