Added another examples for Chapter 9
This commit is contained in:
204
chp9/README.md
204
chp9/README.md
@@ -1,12 +1,20 @@
|
|||||||
# Chapter 9 - Memory Management
|
# Chapter 9 - Memory Management
|
||||||
|
|
||||||
This document summarizes the key concepts from **Chapter 9: Memory Management** in *Linux System Programming, 2nd Edition* by Robert Love.
|
This document summarizes the key concepts from **Chapter 9: Memory Management** in *Linux System Programming, 2nd Edition* by Robert Love*. It provides an overview of how memory is managed in user space, the role of the kernel, common functions for allocation and manipulation, and techniques to optimize and debug memory usage.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Memory management in Linux involves allocating, using, and freeing memory efficiently while avoiding leaks and fragmentation. This chapter explains how user-space programs interact with the kernel’s memory manager through system calls, library functions, and advanced allocation techniques.
|
Memory management in Linux is a fundamental component of system programming. It involves allocating, using, and releasing memory safely and efficiently. Proper memory handling avoids fragmentation, memory leaks, and undefined behavior.
|
||||||
|
|
||||||
|
The chapter focuses on:
|
||||||
|
|
||||||
|
- How user-space processes interact with the kernel for memory allocation.
|
||||||
|
- The process memory layout and its components.
|
||||||
|
- Allocation strategies in both heap and stack.
|
||||||
|
- Advanced techniques like memory mapping and page locking.
|
||||||
|
- Debugging tools for memory issues.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -14,91 +22,191 @@ Memory management in Linux involves allocating, using, and freeing memory effici
|
|||||||
|
|
||||||
### 1. Memory Layout of a Process
|
### 1. Memory Layout of a Process
|
||||||
|
|
||||||
A typical Linux process memory layout includes:
|
Every Linux process has a well-defined memory layout, divided into segments:
|
||||||
|
|
||||||
- **Text Segment** – Executable code of the program (read-only).
|
- **Text Segment** – Holds the compiled machine code of the program. Typically marked read-only and executable to prevent accidental modification.
|
||||||
- **Data Segment** – Initialized global and static variables.
|
- **Data Segment** – Contains initialized global and static variables.
|
||||||
- **BSS Segment** – Uninitialized global and static variables.
|
- **BSS Segment** – Holds uninitialized global and static variables, which the kernel initializes to zero at program start.
|
||||||
- **Heap** – Dynamically allocated memory (via `malloc`, `calloc`, `realloc`).
|
- **Heap** – Dynamic memory allocated at runtime using `malloc`, `calloc`, or `realloc`. Grows upward as needed.
|
||||||
- **Stack** – Function call frames, local variables.
|
- **Stack** – Stores function call frames, local variables, and return addresses. Grows downward on most architectures.
|
||||||
- **Memory Mappings** – Shared libraries, `mmap` allocations, etc.
|
- **Memory Mappings** – Regions created with `mmap()`, commonly used for shared libraries and large allocations.
|
||||||
|
|
||||||
|
> Understanding this layout is essential for writing efficient, low-level programs and debugging memory-related issues.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 2. Dynamic Memory Allocation
|
### 2. Dynamic Memory Allocation
|
||||||
|
|
||||||
- **malloc(size_t size)** – Allocates memory but leaves it uninitialized.
|
Dynamic allocation functions allow flexible use of memory at runtime:
|
||||||
- **calloc(size_t nmemb, size_t size)** – Allocates and zeroes memory.
|
|
||||||
- **realloc(void *ptr, size_t size)** – Resizes a previously allocated block.
|
- **malloc(size_t size)** – Allocates a memory block of the given size without initialization.
|
||||||
- **free(void *ptr)** – Frees allocated memory.
|
- **calloc(size_t nmemb, size_t size)** – Allocates memory for an array of `nmemb` elements, initializing all bytes to zero.
|
||||||
|
- **realloc(void \*ptr, size_t size)** – Resizes an existing memory block, preserving its contents up to the smaller of the old and new sizes.
|
||||||
|
- **free(void \*ptr)** – Releases previously allocated memory.
|
||||||
|
|
||||||
#### Common pitfalls
|
#### Common pitfalls
|
||||||
|
|
||||||
- **Memory leaks** – Forgetting to free memory.
|
- **Memory leaks** – Occur when allocated memory is not freed before losing all references to it.
|
||||||
- **Double free** – Calling `free` twice on the same pointer.
|
- **Double free** – Freeing the same memory block twice, which causes undefined behavior.
|
||||||
- **Use-after-free** – Accessing memory after it has been freed.
|
- **Use-after-free** – Accessing a memory block after it has been freed, leading to crashes or data corruption.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3. `brk` and `sbrk`
|
### 3. `brk` and `sbrk`
|
||||||
|
|
||||||
- Low-level system calls to manage the program break (end of the data segment).
|
Historically, `brk()` and `sbrk()` were used to manipulate the program break, which defines the end of the process’s data segment.
|
||||||
- Rarely used directly; `malloc` and friends handle these internally.
|
|
||||||
|
- `brk()` sets the end of the data segment.
|
||||||
|
- `sbrk()` increments (or decrements) the break value.
|
||||||
|
|
||||||
|
Modern applications rarely use these functions directly; instead, `malloc` and related functions manage them internally. These interfaces are considered low-level and non-portable.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 4. Memory Mapping with `mmap`
|
### 4. Memory Mapping with `mmap`
|
||||||
|
|
||||||
- **mmap()** maps files or anonymous memory into the process address space.
|
`mmap()` maps files or anonymous memory into the process’s address space.
|
||||||
- Advantages:
|
|
||||||
- Direct file access without extra copy operations.
|
#### Advantages
|
||||||
- Efficient large memory allocations.
|
|
||||||
- Common use cases:
|
- Eliminates the need for additional copies when reading files.
|
||||||
- Loading large files.
|
- Enables efficient handling of large files and inter-process communication.
|
||||||
- Shared memory between processes.
|
|
||||||
- Paired with `munmap()` to release mappings.
|
#### Typical use cases
|
||||||
|
|
||||||
|
- Memory-mapped file I/O.
|
||||||
|
- Shared memory regions for IPC.
|
||||||
|
- Allocating large anonymous memory regions.
|
||||||
|
|
||||||
|
To release mappings, use **munmap()**. Unlike `malloc`, which uses the heap, `mmap` provides greater flexibility for aligning memory and mapping hardware devices.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 5. Memory Locking
|
### 5. Advanced Memory Techniques
|
||||||
|
|
||||||
- **mlock() / mlockall()** – Lock memory pages into RAM to avoid swapping.
|
Several specialized APIs provide advanced control over memory:
|
||||||
- Useful for:
|
|
||||||
- Real-time applications.
|
- **mprotect()** – Changes access permissions (read, write, execute) for a memory region.
|
||||||
- Security-sensitive data (e.g., cryptographic keys).
|
- **shm_open() / shm_unlink()** – Creates and removes POSIX shared memory objects for inter-process communication.
|
||||||
|
- **posix_memalign()** – Allocates memory aligned to a specific boundary, useful for SIMD operations or page-aligned buffers.
|
||||||
|
|
||||||
|
These functions are essential in performance-sensitive and real-time systems.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 6. Advanced Memory Techniques
|
### 6. Debugging Memory Issues
|
||||||
|
|
||||||
- **mprotect()** – Change protection (read, write, execute) of memory regions.
|
The GNU C Library provides several functions to inspect and analyze memory usage at runtime:
|
||||||
- **shm_open() / shm_unlink()** – POSIX shared memory objects.
|
|
||||||
- **posix_memalign()** – Allocate memory aligned to a specified boundary.
|
- **mallinfo()**
|
||||||
|
Returns a `struct mallinfo` with statistics about memory allocation, such as total allocated space, number of free blocks, and fragmentation levels.
|
||||||
|
- **malloc_usable_size(void *ptr)**
|
||||||
|
Returns the actual usable size of an allocated block. This value may be larger than the size requested from `malloc()` due to alignment or internal overhead.
|
||||||
|
- **malloc_trim(size_t pad)**
|
||||||
|
Attempts to return unused memory from the heap back to the operating system, retaining `pad` bytes for future allocations. Useful for long-running processes to reduce memory footprint.
|
||||||
|
- **malloc_stats()**
|
||||||
|
Prints detailed statistics about memory allocation directly to `stderr`.
|
||||||
|
This includes information about the heap, allocated blocks, and free space.
|
||||||
|
|
||||||
|
#### Notes
|
||||||
|
|
||||||
|
- These functions are **GNU-specific** and may not be portable across different C libraries or UNIX-like systems.
|
||||||
|
- They are primarily intended for **debugging and profiling**, not for production code paths that require high performance.
|
||||||
|
- Use them to analyze fragmentation patterns, memory usage trends, and allocator behavior during development or testing.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 7. Debugging Memory Issues
|
### 7. Stack-Based Allocation
|
||||||
|
|
||||||
- Tools like **valgrind** and **AddressSanitizer** help detect:
|
Stack allocation can be an efficient alternative for temporary buffers:
|
||||||
- Memory leaks.
|
|
||||||
- Invalid reads/writes.
|
- **alloca(size_t size)** – Allocates memory on the calling function’s stack frame. The allocated memory is automatically released when the function returns, so `free()` is unnecessary.
|
||||||
- Use-after-free errors.
|
|
||||||
- Good practices:
|
#### Advantages
|
||||||
- Always initialize pointers.
|
|
||||||
- Free resources in reverse order of allocation.
|
- Extremely fast allocation and deallocation.
|
||||||
|
- Ideal for short-lived buffers with known size limits.
|
||||||
|
|
||||||
|
#### Limitations
|
||||||
|
|
||||||
|
- Memory is valid only within the function scope. Returning a pointer to stack memory results in undefined behavior.
|
||||||
|
- Excessive allocation can lead to **stack overflow** and program crashes.
|
||||||
|
- `alloca()` is not part of the ISO C standard, but it is available as a POSIX or compiler-specific extension.
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
|
||||||
|
```c
|
||||||
|
void example(size_t size) {
|
||||||
|
char *buffer = alloca(size);
|
||||||
|
// Use buffer safely within this function only
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 8. Manipulating Memory
|
||||||
|
|
||||||
|
C provides several functions for low-level memory manipulation:
|
||||||
|
|
||||||
|
- **memset(void \*s, int c, size_t n)**
|
||||||
|
Fills a block of memory with a given byte value.
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```c
|
||||||
|
memset(buffer, 0, sizeof(buffer)); // Zero out buffer
|
||||||
|
```
|
||||||
|
|
||||||
|
- **memcpy(void \*dest, const void \*src, size_t n)**
|
||||||
|
Copies `n` bytes from `src` to `dest`. Behavior is undefined if regions overlap.
|
||||||
|
|
||||||
|
- **memmove(void \*dest, const void \*src, size_t n)**
|
||||||
|
Like `memcpy`, but handles overlapping memory correctly.
|
||||||
|
|
||||||
|
- **memcmp(const void \*s1, const void \*s2, size_t n)**
|
||||||
|
Compares two memory regions byte by byte, returning:
|
||||||
|
- `< 0` if `s1` < `s2`
|
||||||
|
- `0` if equal
|
||||||
|
- `> 0` if `s1` > `s2`
|
||||||
|
|
||||||
|
- **memchr(const void \*s, int c, size_t n)**
|
||||||
|
Scans the n bytes of memory pointed at by s for the character c, which is interpreted as an unsigned char
|
||||||
|
|
||||||
|
- **memmem(const void \*haystack, size_t haystacklen, const void \*needle, size_t needlelen)**
|
||||||
|
Returns a pointer to the first occurrence of the subblock needle, of length needlelen bytes, within the block of memory haystack, of length haystacklen bytes.
|
||||||
|
If the function does not find needle in haystack, it returns NULL.
|
||||||
|
|
||||||
|
> This function is a GNU extension
|
||||||
|
|
||||||
|
These functions do not perform type checking or add null terminators, making them ideal for raw data operations but requiring caution to avoid buffer overruns.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 9. Locking Memory
|
||||||
|
|
||||||
|
Linux provides mechanisms to lock memory pages into RAM:
|
||||||
|
|
||||||
|
- **mlock()** – Locks a specific memory range, preventing it from being swapped out.
|
||||||
|
- **mlockall()** – Locks all pages mapped by the calling process.
|
||||||
|
|
||||||
|
#### Use cases
|
||||||
|
|
||||||
|
- Real-time systems that must avoid page faults.
|
||||||
|
- Applications handling sensitive data such as encryption keys, ensuring they never hit disk.
|
||||||
|
|
||||||
|
After use, memory should be unlocked with **munlock()** or **munlockall()**.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Key Takeaways
|
## Key Takeaways
|
||||||
|
|
||||||
- Understand the process memory layout to avoid common pitfalls.
|
- Understanding the process memory layout is critical for writing efficient and safe programs.
|
||||||
- Use the right allocation function for your needs.
|
- Dynamic allocation with `malloc` and its variants is powerful but requires careful management to avoid leaks and corruption.
|
||||||
- Prefer `mmap` for large or shared allocations.
|
- Use `mmap` for large or shared allocations and `alloca` for small, temporary buffers.
|
||||||
- Memory debugging tools are essential for writing reliable code.
|
- Memory manipulation functions provide high performance but require strict attention to bounds and alignment.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
- *Linux System Programming, 2nd Edition* – Robert Love, O’Reilly Media.
|
- *Linux System Programming, 2nd Edition* – Robert Love, O’Reilly Media.
|
||||||
- `man malloc`, `man mmap`, `man mprotect`, `man mlock`
|
- `man malloc`, `man mmap`, `man alloca`, `man mprotect`, `man mlock`.
|
||||||
|
|||||||
12
chp9/alignviolation.c
Normal file
12
chp9/alignviolation.c
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
#include <stdio.h>
|
||||||
|
|
||||||
|
int main(void) {
|
||||||
|
char greeting[] = "Ahoy Matey";
|
||||||
|
char *c = greeting[1];
|
||||||
|
unsigned long badnews = *(unsigned long *)c;
|
||||||
|
|
||||||
|
printf("%s\n", c);
|
||||||
|
printf("%lu\n", badnews);
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
36
chp9/mallstats.c
Normal file
36
chp9/mallstats.c
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <malloc.h>
|
||||||
|
#include <string.h>
|
||||||
|
|
||||||
|
int main(void) {
|
||||||
|
/* Initialize string array */
|
||||||
|
char **names = (char **)calloc(5, sizeof(char *));
|
||||||
|
for (int i = 0; i < 5; i++) {
|
||||||
|
names[i] = (char *)malloc(sizeof(char *));
|
||||||
|
}
|
||||||
|
|
||||||
|
memcpy(names[0], "Fabio", 5);
|
||||||
|
memcpy(names[1], "Chiara", 6);
|
||||||
|
memcpy(names[2], "Valerio", 7);
|
||||||
|
memcpy(names[3], "Luca", 4);
|
||||||
|
memcpy(names[4], "Leila", 5);
|
||||||
|
|
||||||
|
printf("\nBefore freed array...\n");
|
||||||
|
malloc_stats();
|
||||||
|
printf("\n");
|
||||||
|
|
||||||
|
for (int i = 0; i < 5; i++) {
|
||||||
|
printf("%s\n", names[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (int i = 0; i < 5; i++) {
|
||||||
|
free(names[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
printf("\nAfter freed array...\n");
|
||||||
|
malloc_stats();
|
||||||
|
printf("\n");
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
62
chp9/stackallocation.c
Normal file
62
chp9/stackallocation.c
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
#include <fcntl.h>
|
||||||
|
#include <errno.h>
|
||||||
|
#include <alloca.h>
|
||||||
|
|
||||||
|
#define MAX_LEN 1024
|
||||||
|
#define SYSCONF_DIR "/etc"
|
||||||
|
|
||||||
|
int open_sysconf(const char *, int, int);
|
||||||
|
|
||||||
|
int main(void)
|
||||||
|
{
|
||||||
|
int fd = open_sysconf("/passwd", O_RDONLY, 0644);
|
||||||
|
if (fd == -1) {
|
||||||
|
perror("open");
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
int len;
|
||||||
|
if ((len = lseek(fd, 0, SEEK_END)) == -1) {
|
||||||
|
perror("lseek");
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (lseek(fd, 0, SEEK_SET) == -1) {
|
||||||
|
perror("lseek");
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
char buffer[len + 1];
|
||||||
|
memset(buffer, 0, sizeof(buffer));
|
||||||
|
|
||||||
|
ssize_t ret;
|
||||||
|
size_t bytes_read_total = 0;
|
||||||
|
while (bytes_read_total < len && (ret = read(fd, buffer + bytes_read_total, len - bytes_read_total)) != 0) {
|
||||||
|
if (ret == -1) {
|
||||||
|
if (errno == EINTR) continue;
|
||||||
|
perror("read");
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
bytes_read_total += ret;
|
||||||
|
}
|
||||||
|
|
||||||
|
buffer[bytes_read_total] = '\0';
|
||||||
|
printf("%s", buffer);
|
||||||
|
|
||||||
|
close(fd);
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
int open_sysconf(const char *file, int flags, int mode)
|
||||||
|
{
|
||||||
|
const char *etc = SYSCONF_DIR;
|
||||||
|
char *name = alloca(strlen(etc) + strlen(file) + 1);
|
||||||
|
strcpy(name, etc);
|
||||||
|
strcat(name, file);
|
||||||
|
return open(name, flags, mode);
|
||||||
|
}
|
||||||
16
chp9/test_malloc.c
Normal file
16
chp9/test_malloc.c
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <malloc.h>
|
||||||
|
|
||||||
|
int main(void) {
|
||||||
|
char *buf = (char *)malloc(256);
|
||||||
|
if (!buf) {
|
||||||
|
perror("malloc");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t size = malloc_usable_size(buf);
|
||||||
|
printf("%d\n", size);
|
||||||
|
free(buf);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user