 > hipl.1.0.1.rar > HACKING
ABOUT ===== This file contains developer information on policies in the HIPL project. TODO ==== - xx AUTOCONF AND AUTOMAKE ===================== Autoconf and automake tools help developers to avoid from "include header dependency hell" so they are also used by HIPL in "tools" and "test" directories. Standard linux kernel supports only plain old Makefiles, so "linux" subdirectory is not controlled with autoconf/automake. The "doc" directory uses plain old Makefile because it uses the same documentation building system as "linux". Libinet6 has been imported from the Usagi project and uses only automake (no autoconf). If you need to add or remove source/header files or modify build rules, edit "Makefile.am". If you need to modify dependencies, edit "configure.ac". After editing the files, run "autoreconf". Autoreconf may then update some files (config.h.in, config.sub, configure, Makefile, Makefile.in) and you need to commit them too into the CVS along with your new source and header files. If you need to share a file between userspace and kernelspace, place it in the kernelspace and modify userspace Makefile.am to create a link to it from userspace (CVS does not accept links). For example, "tools/Makefile.am" makes a local link to "hipl/linux/net/ipv6/builder.c" by the following rule: builder.c: $(LN_S) ../linux/net/ipv6/hip/builder.c . CODING CONVENTIONS ================== - Don't invent your own coding style. It will give the code an ununified look. Especially, if you don't follow the memory deallocation procedure below, you might even cause some memory leaks. - All kernelspace code should follow the standard linux coding conventions listed in
/Documentation/CodingStyle - All code shared between userspace and kernel should follow kernel coding style. - HIPL code should use the prefix "hip_" in all function and global variable definitions to avoid namespace pollution, especially in the libraries. The prefix is not necessary in userspace code, but the same naming conventions might be clearer to use anyway. - The width of one line should be limited to 80 characters in userspace code. This rule may be relaxed in the kernel code because the space is very limited due to the 8 character indentation. - All functions should return an error value instead of "ok" value. That, is zero for success and non-zero for failure. - Use the HIP_IFE, HIP_IFEL macros to reduce the number of lines of code. - Do not make unreadable code by nesting code too much: if (foo) { if (bar) { if(hello) { confuse_me(); } } } Instead, join the conditional expressions (foo && bar && hello) or use goto (goto skip;). - Declare variables right after beginning brace, not in the middle of the code: int foo { int i; /* integers declared in the beginning of the function */ char c; bar(); for (i = 0; i < 100; i++) { int x; /* This is fine because it is after the beginning brace of loop */ hello(i, x); } } hipsock shares code with the rest of the code and it requires ISO C90 style C code. Declare variables right after braces! - Memory (de)allocation procedure is as follows: int f() { char *mem = NULL; HIP_IFEL(!(mem = HIP_ALLOC(256, 0)), -1, "alloc\n"); out_err: if (mem) free(mem); return err; } QUICK-AND-DIRTY ARCH INSTRUCTIONS FOR DEVELOPERS ================================================ These instructions assume that you want to access the HIPL source code using ssh. Note: tla and bazaar provide the same functionality but bazaar was faster and more user friendly when this was written. # Register an account: # baz register-archive hipl-dev@freelists.org--hipl \ s baz my-default-archive hipl-dev@freelists.org--hipl # Get a working copy from the main branch: # baz get hipl--main--2.6 hipl--main--2.6 # Copy the tla hook if you want to get your commit logs to the mailing list: # cp hipl--main--2.6/test/tla-hook ~/.arch-params/hook # Different ways to commit code: # a) baz commit # Open's your default editor and commits. b) emacs `tla make-log`; tla commit c) baz commit -s "Explanation jaada jaada" # How to check what your have modified in the tree # baz changes # How to make a branch from the current version of the main branch # baz my-default archive # Make sure that you are accessing the HIPL archive baz archive-setup hipl--mybranch--2.6 baz tag hipl--main--2.6 hipl--mybranch--2.6 # Get the new branch # baz get hipl--mybranch--2.6 hipl--mybranch--2.6 # How to synchronize branches... remember to make clean commits, i.e. do # NOT commit a merge and local modifications in the same commit! # cd hipl--mybranch--2.6 baz changes # Check that the tree is unmodified baz star-merge hipl--main--2.6 --- Optional --- Below are some instructions on using revision libraries. They speed-up tla/baz operations. # Make a revision library (otherwise e.g. commit and changes take up to # 15 mins!) # mkdir ~/.tla-rev-lib baz my-revision-library ~/.tla-rev-lib/ baz library-config --greedy --sparse ~/.tla-rev-lib/ # Get a working copy from the main branch using revision libraries: # baz get --link hipl--main--2.6 hipl--main--2.6 Also, the tla usage is optimized using hard-linked revision libraries. # Run the command below in your working copy (e.g. monthly) if you # experience a serious slow-down in the commit speed or you want to enable # revision libraries in your local copy: # cd hipl--main--2.6 baz changes --link ARCH POLICY =========== *** Important: see that your umask is 0002 before you make any commits *** hipl--main--2.6 is the "stable" branch for HIPL code. Do not put any experimental code in there because other people are relying on that the code works! Commit only minor bugfixes directly to the stable branch, and use/create other branches to develop experimental code. When you have tested and are sure that code works, you can merge it into the stable branch. Some points in the branch quality assurance: - the code must compile as it is without errors - compiler warning messages must be taken seriously and must not be ignored unless you really know what the message means. And we REALLY mean really. - the base exchange should be manually tested between two hosts - UPDATE packet exchange should work - the code should not leak memory or do bad locking - (all existing unit tests should be run without any errors (see TESTING)) DEBUGGING ========= Debugging the kernel with gdb is not possible unless you're running User Mode Linux and in such cases you may prefer manual debugging statements in the code. It may be also your personal favourite to prefer debugging statements over gdb in the userspace. HIPL provides a set of wrappers for adding debugging statements in a concise way: - HIP_DIE(arguments as for printk) - HIP_ERROR(arguments as for printk) - HIP_INFO(arguments as for printk) - HIP_DEBUG(arguments as for printk) The wrappers exists for many reasons. First, it is very convinient to have the same syntax for the debug statements indepently of whether you degugging in kernel or userspace. Second, an "ad hoc" debug statement mechanism would make it more difficult to adopt HIPL into production environment because it would be awkward to comment all the superfluous debug messages. The superfluous debug messages can be suppressed by a single switch to configure (CONFIG_HIP_DEBUG) if an unified interface for debugging is used. Third reason for the existance of wrappers is that a uniform importancy level of debug messages can be enforced. For example, error messages are more important than info messages and info messages are more important than debug messages. One benefit of the importancy of levels is that the debug messages can be cathegorized into two groups: the ones that are always included in a build and the ones that are included only in a development build. HIP_DEBUG belongs to the latter group and the other functions belong to the second group. The fourth reason for justifying the wrappers is that some programs cannot be run interactively on the screen in production environment. For example, the HIP daemon is preferred to be run as a background process in production environment by the end-users. The debug statements cannot be printed on the screen in such a case; instead, the statements have to passed via syslog. On the other hand, the developer wants to run the daemon interactively. The wrappers make it possible to please both of the groups easily. Debug statements should not be removed from code because someone may have use for the debug statement later on. The preferred way is to put a "_" in front of the debug function name to prevent the debug statement to ever to be shown (indepently of whether the build is a development or production build): - _HIP_DIE(..) - _HIP_ERROR(..) - _HIP_INFO(.. - _HIP_DEBUG(..) DOCUMENTATION ============= Doxygen is now used for documenting HIPL. Manual for how to use doxygen documentation, is found from here: To create doxygen documentation in HTML format, execute "make doxygen". Doxygen will create documentation under doc/doxy/html. Simple example, how to use some doxygen features: /** Description of a #define. */ #define HIP_ENABLED 1 /** * Description of a global variable. * This can continue also to multiple lines, * just like previous #define description. */ int hip_sock = -1; /** * This is a function description. Parameter documentation is in * form "@param description". Return value is documented * using "@return description". Todos are documented using * "/*! \todo description */". Then the todos are listed in doxygen * created documentation as list. * * @param name This is a function parameter description. * @return This is a return value description. */ int test(char *name) { /*! \todo * Make contents for this function!!! */ } SUBMITTING PATCHES ================== HIPL is still work in progress. The code is being constantly developed and it may be difficult to merge your contribution to the HIPL code. If you still want to submit patches to the HIPL project, see that the patch follows the CVS POLICY (see above). Build your patches against the HEAD branch in CVS. It would be nice if you would build one (or multiple) unit test case to validate your patch. TESTING ======= The project has a tailored unit testing environment which makes it possible to run tests in the HIP kernel module and userspace. It is recommended that the developer builds a new test case when creating new functionality or repairing old functionality. Still, there are some tangled functionality, especially in the kernel, that is very difficult to test with unit testing. Manual black box testing should be used to cover changes in the code in such cases. CREATING A TARBALL OF THE SOURCE CODE ===================================== There is a "make dist" target which builds a tarball of the hipl source code. It includes only the files which are necessary. The included files are specified in SOURCES and HEADER rules in */Makefile.am, but it is also possible to force some extra files in top most Makefile.am. PREBUILT BINARY PACKAGES ======================== Redhat ------ Note well: Currently the status of the packaging scripts is in very initial phase. They are not tested throughly. Two separate packages are planned to be provided by HIPL: one for user space software and the other for kernel and kernel modules. Currently only scripts for building user space packages exist. There are scripts in directory test/packaging that can be used to create binary packages. The aim for these packages is for easier testing of HIPL software. With the scripts, RPM (e.g. for Fedora Core) and DEB (for Debian) packages can be created. To create the RPM package, type: "./autogen && ./configure && make rpm". This will create a source package (in tar.gz form in the same directory) which will be used when building the RPM file. After the script has run successfully, follow the instructions given by the script. That is, login as root and copy the source package (e.g. hipl-userspace-1.0-1.tgz) to directory /usr/src/redhat/SOURCES. Then issue the command "rpmbuild -ba /test/packaging/hiplx.spec". When everyting is finished successfully, there will be a binary RPM package "/usr/src/rpm/RPMS/i386/hipl-userspace-1.0-1.i386.rpm". The package can then be installed with rpm -i. For more info about rpm packaging, please see: Debian ------ DEB package is created similarly: "./autogen && ./configure && make deb". The result is both binary and source debian packages: - Binary Binary Debian package can be found in test/packaging/ directory with the following name: hipl-1.0-1_i386.deb. The package can be installed using dpkg, e.g. "dpkg -i hipl-1.0-1_i386.deb". - Source Debian source package is made of three files which are: hipl-1.0.diff.gz hipl-1.0.dsc hipl-1.0.orig.tar.gz and can be found under directory test/packaging/hipl-1.0-deb-src/ HOW TO CREATE KERNEL PACKAGES ============================= Debian/Ubuntu ------------- To create the package from vanilla sources for Debian distribution or any distributions based on Debian (e.g. Ubuntu Dapper Drake), follow the following instructions. * Download and extract the vanilla kernel source (I got it from () I extract it to /usr/src and I added my username to src group by typing adduser my_username src * cd linux-2.6.xx * Patch the kernel with the hip patches you can find under hipl/patches directory * cp /boot/config-something .config In order to build the binary package type the following: * fakeroot make-kpkg --initrd --append-to-version=.hipl --revision=custom.1.0 kernel_image Instead, if you would like to produces a debianised package of the Linux kernel sources type the following: * fakeroot make-kpkg --initrd --append-to-version=.hipl --revision=custom.1.0 kernel_source After the kernel has been successfully compiled, you will find the debian packages in the top directory. To install it type: dpkg -i kernel-image-2.6.16.5.hipl_1.0_i386.deb and reboot. The debian package should already include the new installed kernel for grub. If you need grub check this: Main reference (.) Redhat notes ------------ The instructions below did not work anymore. Instead, I followed these instructions: see: test/packaging/kernel-2.6.17.14.spec To create the package from vanilla sources (preferred method) for Fedora, follow the instructions of this document: In short: * download and extract kernel sources * cd linux-2.6.xx * apply hip patches * cp /boot/config-something .config * make rpm # or bin-rpm?? To install, do this: * rpm -i /usr/src/redhat/RPMS/i386/kernel-2.6.xx.rpm * /sbin/new-kernel-pkg --mkinitrd --depmod --install --make-default 2.6.xx NEW-KERNEL-PKG IS NECESSARY AS THE RPM DOES NOT INCLUDE INITRD AND THE KERNEL IS NOT INCLUDED IN GRUB CONFIGURATION. That will create and install vanilla rpm. However, if you want to use the Fedora/Redhat patches to improve e.g. hardware support, use the instructions below. To create rpm for Fedora patched kernel, you need to download the kernel from somewhere from here: For RH/Enterprise, you need to have an account on rhn.redhat.com to get the latest packages. Browse/and search the channels list. I basically followed these instructions: This is what I did: * rpm -ivh kernel-2.6.10-1.760_FC3.src.rpm * I modified /usr/src/redhat/SPECS/kernel-2.6.spec to include the HIP patches: %define rhbsys .beet # HIPL patches Patch190: simple-beet-ph-patch-v1.0-2.6.16.5 Patch191: interfamily-beet-ph-patch-v1.0-2.6.16.5 Patch192: policy-sleep-2.6.16.5-v2.patch Patch193: hipmod-2.6.16.5-v1.patch # HIP patches %patch190 -p1 %patch191 -p1 %patch192 -p1 %patch193 -p1 * copied the patches to /usr/src/redhat/SOURCES * rpmbuild --target i686 -ba /usr/src/redhat/SPECS/kernel-2.6.16.9.spec UPGRADING THE KERNEL - DEPRACATED ================================= # note: there is a perl script called "hashline" diff which is more # convinient to use in upgrading the kernel (to check for added/deleted # files) % % cd hipl--fix--2.6/linux % patch -p1 <~/down/patch-2.6.7 2>&1| tee /tmp/PATCH_LOG % grep ^patching /tmp/PATCH_LOG|cut -d' ' -f3 >/tmp/CHANGES % for i in `cat /tmp/CHANGES`; do tla id $i 2>&1; done >/tmp/TLA_LOG % tla add `perl -ne 'print "$_" if ($n); $n=0; $n=1 if (/id: untagged file/)' /tmp/LINUX_DIRS % for i in `cat /tmp/LINUX_DIRS`; do tla id $i 2>&1; done >/tmp/TLA_DIR_LOG % tla add `perl -ne 'print "$_" if ($n); $n=0; $n=1 if (/id: untagged file/)'