Description
Problem description
In this assignment, you will build a mini-DMTCP. DMTCP is a widely used software package for checkpoint-restart: http://dmtcp.sourceforge.net. However, you do not need any prior knowledge of DMTCP for this assignment. The software that you produce will be self-contained.
You will write a shared library, libckpt.so. Given an arbitrary application in C or C++, you will then run it with this library with the following command-line:
LD_PRELOAD=/path/to/libckpt.so ./myprog
where /path/to/libckpt.so must be an absolute pathname for the libckpt.so.
You must provide a Makefile for this project. Here is some advice for writing the code for libckpt.so. However, if you prefer a different design, you are welcome to follow that. Your code will only be tested on single-threaded targets. You are not required to support multi-threaded applications. Similarly, you are not required to support shared memory.
(Note that the memory layout of a process is stored in /proc/PID/maps. The fields such as rwxp can be read as “read”, “write”, “execute” and “private”. Naturally, rwx refer to the permissions. Private refers to private memory. If you see “s” instead of “p”, then it is shared memory.)
Saving a checkpoint image file
Your code should read /proc/self/maps (which is the memory layout of the current process).
Your code should then save that information in a checkpoint image.
You should save a file header, to restore information about the entire process. This should include saving the registers. (See below on how to save and restore register values.)
For each memory section, you should save information that you will need to later restore the memory:
a section header (including address range of that memory, and r/w/x permissions)
the data in that address range. (See below for an easy way to read the hexadecimal numbers used to specify a memory address range.)
We need a simple way to trigger the checkpoint. Write a signal handler for SIGUSR2, that will initiate all of the previous steps.
To trigger a checkpoint, you now just need to do from a command line:
kill -12 PID
where PID is the process id of the target process that you wish to checkpoint. (Signal 12 is SIGUSR2, reserved for end users.
The signal handler will only help you if you can call signal() to register the signal handler. We must write the call to signal() inside libckpt.so. But it must be called before the main() routine of the end user. The solution is to define a constructor function in C inside libckpt.so. For the details of how to do this, see, below: “Declaring constructor functions”
The name of your checkpoint image file must be myckpt.
Restoring from a checkpoint image file
Create a file, myrestart.c, and compile it statically at an address that the target process is not likely to use.
For example:
gcc -static \
-Wl,-Ttext-segment=5000000 -Wl,-Tdata=5100000 -Wl,-Tbss=5200000 \
-o myrestart myrestart.c
Note: The gcc syntax -Wl,Ttext=… means pass on the parameter -Ttext=… to the linker that will be called by gcc. See man ld for the Linux linker. ]
Note: In reading man ld, you’ll see that numbers like 5000000 should be interpreted as hexadecimal.
When myrestart begins executing, it will need to move its stack to some infrequently used address, like 0x5300000. A good way to do that is to:
Map in some new memory for a new stack, for example in the range
0x5300000 – 0x5301000
To create this memory, use mmap(). [ See: man mmap ]
The myrestart.c program should take one argument, the name of the checkpoint image file from which to restore the process. Now copy that filename from the stack (probably from ‘argv[1]’) into your data segment (probably into a global variable). (We will soon be unmapping the initial stack. So, we better first save any important data.) When you copy the filename into allocated storage in your data segment, an easy way is to declare a global variable (e.g., ‘char ckpt_image[1000];’), and then copy to the global variable.
Use the inline assembly syntax of gcc to include code that will switch the stack pointer (syntax: $sp or %sp) from the initial stack to the new stack. Then immediately make a function call to a new function. The new function will then use a call frame in the new stack.
Specifically, you will want some code like:
asm volatile (“mov %0,%%rsp” : : “g” (stack_ptr) : “memory”);
restore_memory();
Note: you will see similar code near the beginning of the function restart_fast_path(): https://github.com/dmtcp/dmtcp/blob/76ca86/src/mtcp/mtcp_restart.c#L398
Unmap the stack used by the myrestart program. The reason is that the stack might be located at an address range that conflicts with what the target program was using prior to checkpointing. Since the goal is to restore the memory as it existed at checkpoint time, the conflicting addresses can lead to hard to debug problems.
To unmap the stack:
read the /proc/self/maps of the myrestart process;
search for the region with the name [stack] denoting the current stack in use; and
call munmap on the stack region.
Copy the registers from the file header of your checkpoint image file into some pre-allocated memory in your data segment.
The memory of your myrestart process should no longer conflict with the memory to be restored from the checkpoint image. Now read the addresses of each memory segment from your checkpoint image file. It’s in your section header. Call mmap to map into memory a corresponding memory section for your new process. (See man mmap)
Copy the data from the memory sections of your checkpoint image into the corresponding memory sections.
Now, you need to jump into the old program and restore the old registers. In a previous step, you had copied the old registers into your data segment. Now use setcontext() to restore those as the working registers.
Note: See man setcontext. Also, note that this will restore your program counter (pc) and your stack pointer (sp). So, you will automatically start using the text segment and stack segment from your previously checkpointed process.
