What is the "Stack Smashing Detected" Error?

• The Stack smashing detected error is caused during the execution of a program when there is a chance for buffer overflow.
• This error is a defensive mechanism used to prevent stack-based buffer overflows.
• The compiler uses a random sequence of bits called canary to check for buffer overflow and based on that, produces the stack smash detected error in C.
• This is a runtime error and leads to the termination of the running program.

A buffer overflow occurs when the size of data to be stored in the memory exceeds the allocated size for the data in the memory. Buffer overflow leads to overwriting of the returns address of the function.

Let us understand what causes this stack-based buffer overflow,

• When a function is called, the return address of the function and the parameters in the function are stored in a stack.
• The stack pointer is used to represent the starting address of the stack after the return address and function parameters have been pushed into the stack.
• The frame pointer(also called base pointer) is initially assigned to the address in the stack pointer.
• The address of the stack pointer decreases as the space is continuously allocated in the stack for local variables in the function. The address decrease because usually a stack grows downwards in memory.
• The frame pointer is used to access and modify all parameters and local variables during the execution of the function.

Now consider a parameter or local variable with an allocated size of 10, but the original size of data is 20.

• A heap is another data structure used to allocate space for data in memory. The heap usually grows upwards. In our example, a heap of size 10 is allocated in the memory.
• As data grows to size 20, the heap expands beyond its allocated size to accommodate the additional data, potentially overwriting return addresses and other data in the memory.
• This causes a stack smashing detected error in C. We will look into the programmatical explanation in the examples section.

Let us understand how the compiler detects the buffer overflow with the help of Canary,

• A canary is a sequence of bytes that is pushed into the stack just after the return address of the function is pushed into the stack by the stack smash protector.
• The canary is also called guard variables because they are used to protect important assets in the stack such as the return address of the function.
• The value of the canary is checked periodically for any change from the initial value. If any change is detected, the stack smashing detected error is produced.
• The –fstack-protector compiler flag can be used to induce the compiler the use canary in functions. There are many types of canaries such as,
  • Null canary - used by strings or character array as termination point(null character)
  • Random canary - used by the GCC compiler on compiling with the –fstack-protector flag or the -fstack-protector-all flag.
  • Terminator canary - used to indicate the end of input while reading in a file or an input stream by the EOF(End of file) marker.
• Canary is not used as default by the compiler as they lead to loss of performance.

A sample output showing the stack smashing detected error is,

How to Fix the Stack Smashing Detected Error in C.

There are different ways to debug and fix the stack-smashing detected error,

• A basic understanding of memory allocation in C can help us prevent this error. We must make sure that the data should not larger than the allocated memory for the data. For example, consider the following should not be done,

The character_array has a size of 10, but the string to be copied is of length 15 and can't be accommodated in the character_array. Therefore this operation results in a buffer overflow.

• The compiler flag, -fno-stack-protector can be used during the compilation of the program to prevent the use of a canary in the stack. This prevents the stack smashing detected error, but the program may still terminate due to a segmentation error which is caused due to illegal access to a memory location. The command used to perform this is,

Here code.c indicates the file with vulnerable code that may lead to a buffer overflow error. A segmentation error is a type of error that occurs when the program tries to access parts of memory that are not allocated for the use of the program.

• We can predict at which memory location the buffer overflow has occurred and what causes it using the google address sanitizer. The google address sanitizer can be used to detect memory errors through debugging and can be enabled during the compilation of the program with the -fsanitize=address flag. The following command is used to compile a program in C with google address sanitizer enabled.

• Prevent the use of vulnerable functions that can cause a buffer overflow and replace them with safer alternatives. For example, the gets() function is considered vulnerable because it allows reading input of a large size that could not be accommodated in the allocated memory. In such cases, we can use the gets_s() function as a safer alternative to the gets() function which restricts the size of the input by specifying the maximum size of input that can be received in the second parameter The syntax of the gets_s function is,

The str is the character pointer or character array to which the input will be stored and n depicts the maximum size of the input. If the size of the input exceeds the value of n, the gets_s() function stops execution and returns an error. This mechanism will prevent buffer overflow.

The rsize_t data type is used to indicate that we are storing the size of a single object.

• We can use the FORTIFY_SOURCE feature to implement buffer overflow checks on vulnerable functions such as gets(), strcpy() etc... The -D_FORTIFY_SOURCE=1 compiler flag is the flag used to enable the FORTIFY_SOURCE feature which performs checking the code for possibilities of buffer overflow in compilation time. This flag is used with the optimization flag -O2 to perform more checking and speed optimization. The following command illustrates how to use the command,

• There is also a way to fix both the stack smashing detected and the segmentation fault error by disabling all the protection mechanisms in the system and allowing the buffer overflow. This should only be used during testing as this will make the system vulnerable to buffer overflow attacks. The command is shown below,

Examples Of Stack Smashing Detected in C

An example code that can cause the buffer overflow is as follows,

In the above example, the strcpy()function is the cause of buffer overflow as the function copies a string of size 15 to an array of allocated size 10. The memory allocated is lesser in size than the data to be copied, hence resulting in a buffer overflow and the stack smashing detected error in C. Let us compile the file buffer_overflow.I'm with the –fstack-protector` flag,

The - o flag is used to indicate the name of the file to which the compiled code is stored. By default, the name of the file is a.out. Let us run the complied code,

On running the code, we can see the stack smashing detected error.

Now, let us compile the code again with the fno-stack-protector flag to prevent the stack smashing detected error.

Let us run the compiled code.

There is no error indicating the occurrence of buffer overflow. There is also no segmentation error because the return address of the address has not been overwritten by the overflow. Now, let us use the google address sanitizer to understand at which memory location the buffer overflow has occurred.

On running the code,

The compiler using the google address sanitizer gives information on the line at which the buffer overflow occurs and which variable is overflowed by the buffer overflow.

Let us use the FORTIFY_SOURCE macros to find the vulnerable function which causes the buffer overflow using the D_FORTIFY_SOURCE=1 and optimization flags,

The function which causes the buffer overflow is indicated with the variable responsible for the buffer overflow on compiling the code. The optimization flag is used to improve the performance of code and display detailed output.

An example of stack smash detected involving the gets() function is,

On compiling this code with the -fno-stack-protector flag,

On running the complied code we get a segmentation error because the return address of the function buffer_overflow can be overwritten by the buffer overflow.

Learn More About Stack in C

• A stack is a data structure used to store the collection of data.
• The data first inserted into the stack can only be removed last and hence stack is called the last in first out data structure.
• Learn more about Stack in C.