Continue reading »]]> It’s been three years since we upgraded the hardware that hosts our various sites. I contacted my provider (Crucial Paradigm) and got some competitive offers. Stefan, my friend in Berlin that I split the server with, and I agreed on the following specs:

• Athlon 64 x2 4000 (2 cores @ 2.1GHz, 512K L2 each)
• 4GB RAM, 160GB SATA, 100Mbps
• CentOS 5
• Apache, BIND, MySQL, Postifx, Spam Assassin, ClamAV, Cyrus IMAP
• SELinux enforcing

Once again this is going to be a dedicated, remote, hosted server. A few days later and they contacted me with the login information. I’m going to describe the move from a high-level. I’m not going to go through the individual config file modifications or how to dump a Cyrus database.

It only took me about a day to prep the new system for the move. Stefan was gone for a week so I could move pretty much all of my stuff but I would wait until he got back to move the shared mail services. It is rather difficult to incrementally move users from one system to another using Cyrus so I just did a test migration of the IMAP spools and databases but waited on actually updating the DNS MX records.

First I updated the glue records and name servers at my registrar. I made the new system master and the old system I just modified to run as a slave. I also had Josh fix his config since he is my DNS buddy (I highly recommend using the DNS buddy system). With DNS up and running I added entries for the new server.

Then I moved my website. I just dumped my databases and pulled them into the new system. A few Apache tweaks and everything was good to go. I was pulling my hair out at one point trying to figure out why the CentOS default page was appearing but I finally tracked it down to a welcome.conf configuration file. I removed that and found I could debug Apache config problems more easily.

I decided to tackle all of the SSL stuff at once. I setup Apache, Cyrus, and Postfix w/ certs and CAs. I tested each service to ensure I had the directory permissions correct and had the authentication mechanisms properly configured. It is at this point that I tested a Cyrus migration knowing that I would be repeating the process again later.

Then I moved on to configuring the rest of the Postfix chain using Amavis, Spam Assassin, and ClamAV. There was plenty of documentation on this process available online. Note that ClamAV (and maybe a few of the others) are only available in the rpmforge yum repos. These programs aren’t officially part of the CentOS/RHEL distributions but are commonly used by the user communities. As a result their configuration files don’t mesh precisely with the rest of the services, for example storing the virus database in /var/clamav by default instead of /var/lib/clamav. I fixed these discrepancies and would recommend you do the same. It did take me a while to track all of these down. freshclam (the ClamAV updater) was running from a Cron job. It was trying to write to “/var/clamav” which didn’t exist. As a result the virus signatures weren’t updating. I discovered this only through reviewing the logs.

As soon as Stefan got back he moved all of his services and data. Finally we cut the mail over to the new system by fixing the MX records and using the Postfix transport map feature on the old system to force it to relay to the new server. That completed the migration.

SELinux
Upon login I discovered SELinux was disabled despite my specific request that it be enabled. I decided to fix it by hand instead of letting Brickwall fix it automatically to show that going from disabled to permissive to enforcing isn’t bad and shouldn’t be scary. It should be done regardless of the presence of a management tool like Brickwall. I changed the flag in /etc/selinux/config to enforcing and ran # touch /.autorelabel.

I also added enforcing=0 to the ends of the kernel command lines in /boot/grub/menu.lst.

Tangent – I chose this route, modifying the grub file, for going into permissive mode during configuration and testing instead of setting the flag in /etc/selinux/config to permissive. Applications, such as those built into Red Hat and CentOS as well as third party applications like Brickwall, modify the settings in /etc/selinux/config. It is possible for me to inadvertently set the system to boot into enforcing before I am ready using these applications. Since this is a remote server and the configuration is in flux I want to make sure a quick remote reboot request gets me back in to my system. It is the exact same as using iptables remotely; you don’t load the firewall rules automatically on boot until you’re sure you can get back in through SSH. As soon as I’m confident that I typed in the SSH network settings properly the system boots into enforcing mode.

After setup is complete remember to remove the kernel command line flag – always going into permissive mode on reboot, much like not auto-loading firewall rules, is a patentable bad idea on a production system.

I disabled all unnecessary services (everything except SSH at this point) and rebooted. The system relabeled and came up in permissive mode. Everything was working fine and running in the proper SELinux domains according to # ps axZ. I ran # setenforce 1 to go into enforcing mode for the first time.

Nothing bad happened. The world didn’t end. The system didn’t stop responding. I’m also assuming the rampant rumor that SELinux kills Giraffes in enforcing mode is false because I watched the news and saw nothing Giraffe related. This SELinux stuff really isn’t as hard, or as mean as people make it sound. BTW Brickwall would have taken care of all of this editing of SELinux config files and relabeling stuff for me if I wasn’t such a control freak.

Well turning it on is one thing. Really using it is a whole ‘nother thing right? That is where Brickwall comes into play (disclaimer [1]). I mentioned it before but think “SELinux Management” and there are free versions for Fedora and demos for RHEL and CentOS. Because I’m cool I get to use the Enterprise Edition. Mere mortals may have to run Professional or Standard via “ssh -X”. There is no difference in configurability. Both use the same configuration GUI. The only policy difference is the policy for the remote daemon – not really needed if the remote enterprise management daemon isn’t installed. One has centralized/remote management, the other does not.

Brickwall Enterprise Edition has two components, the centralized manager application with configuration editor and the remote daemon. The centralized management application takes different plug-ins – I’m only going to be using Brickwall plug-in that supports general SELinux configuration. I installed the Enterprise Manager with the Brickwall Enterprise component on my desktop RHEL 5 system. This installation process generated the remote daemon RPMs for me. These packages include SSL keys tying the remote daemon to this specific Enterprise Manager install. The SSL keys are used to encrypt the network traffic between the manager and the client daemons.

I installed the remote daemon on the new system. It is just a little daemon that facilitates remote management of SELinux policy. I added one of the IPs of this system to a group in my Enterprise Manager. The system had to be “activated” which means Brickwall had to switch from the standard targeted policy to a Brickwall policy. There aren’t really any functional differences between the two policies – a default Brickwall policy is semantically equivalent to a targeted policy as shipped by Red Hat/CentOS. But the Brickwall policy contains the structure we need in-place to customize the policy later.

Continuing w/ Brickwall I started restricting network settings for the services I would be running (listed above). I restricted things like the spam, virus, mysql, and other mail filtering services to local host only. I restricted service ports to meet my requirements. I applied the configuration changes and rebooted and went into enforcing to verify services came up in the proper domain after a reboot.

All of the SELinux domain names show exactly what processes they target so run`ps axZ`. All of you key services should be running as “*:*:servicename_t”, such as “mysqld_t” or “httpd_t” or “postfix_smtp_t”.

The only thing that should be “unconfined_t” is user processes or custom services that aren’t targeted by SELinux. Note that there are some 200 odd applications or services covered by SELinux so there is a good chance your “custom service” is covered.

[1] Disclaimer: I work for the company that makes Brickwall. Since I’m only an amateur blogger (not commerically endorsed, still eligible for the Bloglympics) and since this doesn’t appear on tresys.com I’m not going to be writing a review or praise the software, I’m just going to run through using it to batten down the hatches on my system.

Continue reading »]]> Lately conversations keep turning up new slogans for SELinux. I figure this is as good a place as any to keep a running list so here we go:

• SELinux – Because users do weird shit.
• SELinux – Fuck root.
• SELinux – Hampering administrators since before it was cool.
• SELinux – Take revenge against the BOFH
• SELinux – High-security gone haywire.
• SELinux – Turning it off is like removing the batteries from a smoke detector. Sure it sounds better but you might get burned.
• SELinux – Because life is too simple.
• SELinux – AppArmor sucks.
• SELinux – It’s too early in the morning to be cleaning up after 11-year old kiddies.
• SELinux – Too powerful for our own good.
• SELinux – Here’s our root password, what’s yours?
• SELinux – Didn’t they teach you about using protection in high-school?
• SELinux – Blind faith not required

Thinking about slogans actually got me thinking about “short reasons to use SELinux”.

• SELinux will save you tons of money, your TCO will go down and your ROI will go up.
• SELinux supports 3-letter acronyms out of the box, no complex policy changes required.
• Zero day vulnerabilities are a fact. Do something about it.
• Trusted Solaris has been end-of-lifed and you’re not in the government space to begin with.
• Path-named based access control is weak.
• Implicitly trusting admins doesn’t have to be SOP.
• You’re not a security expert, let us do the hard work.
• The US military (and others) trust SELinux with their information, shouldn’t you? [1]

These are just a few.

[1] The answer to this question might actually be a resounding “no!” Don’t worry, I’m not offended.

Continue reading »]]> We in the security community have been trying to explain the benefit of MAC to developers in the embedded device arena for awhile now. Maybe if people keep threatening devices with radio chips and tons of proprietary crap the embedded developers will jump onboard. Motorola has been using SELinux/SEBSD on the A1200 and other devices for awhile now. Given the high-level view of the policy and lack of knowledge about the proprietary software and architecture driving the device I can’t really jump to any conclusions about the completeness of their policy but they are at least trying.

The reason I’m mentioning this is the most recent tiff exploit in the iPhone. Some hackers (the good kind) got ahold of it and used it to slip Installer.app on the iPhone. They were even kind enough to close the door they come in through; the exploit patches the vulnerability after it has installed Install.app.

I know this is SELinux and not SEBSD, although these permissions are in both IIRC, but lets try to count the permissions SELinux would have at its disposal to stop such an “attack”:

• memory protection
• file write protection (although Erica Sadun says this is done by “reassigning the root of the file tree”, whatever the hell that means… chroots? bind mounts? namespaces?)
• execute
• execute_no_trans

Listing these permissions is great, but what if Apple had needed Safari to download and install updates? Perhaps they could use a small app trusted to verify that Apple had signed the apps. Protect this app and it’s resources via SELinux and you’re golden.

I believe that DRM and security are hard to enforce when the keys are in the hands of the users. Someone will likely figure out how to view the filesystem or at least how to access it by sniffing USB traffic or looking at the hardware being used (in the case of DVD players etc). In this case, if you created a kernel that refused to boot in permissive mode would that be sufficient? Perhaps not, they could feasibly load their own kernel but that would be much more difficult given a properly architected MAC policy protecting the kernel itself.

Continue reading »]]> I reckon about three people a day enter the #selinux channel on freenode, ask a question, and than leave a few minutes later without giving anyone a chance to respond. Since no question askers read the topics or have the time to idle I figured I’d start posting their questions here. There is a good chance I won’t be able to answer them without more detail, but hell, its gotta be better than nothing (maybe). This quote is from the IRC channel so forgive the formatting.

how can i give a user read access to the /etc/mail (etc_mail_t) sendmail.cf?
when i try to connect to sendmail: NOQUEUE: SYSERR(rattler):
/etc/mail/sendmail.cf: line 0: cannot open: Permission denied


On a targeted system a “user”, as in the traditional Unix sense, is not confined in any manner. On a default FC6 install I can read /etc/mail/sendmail.cf with no denials in enforcing mode with any user logged in at the local console or via SSH.

So given the fact that any user can read the file I can only assume you mean the daemon itself. Well, once again on a default system, the sendmail daemon can read the file you specified. On my system the file has the type:

[spencer@sshimko-fc6 ~]$ ls -Z /etc/mail/sendmail.cf
-rw-r--r-- root root system_u:object_r:etc_mail_t:s0 /etc/mail/sendmail.cf


Make sure your file is labeled similarly. If not, run “restorecon /etc/mail/sendmail.cf” and restart the mail service. If this still doesn’t work could you please give more details?

Continue reading »]]> OK I’m going to go ahead and post this in the hopes it forces me to finish the series. Check back for updates. Not going to be this weekend, but by next weekend I promise. I’ve had the thoughts saved since I started part 1, but things kinda went awry (marriage, and then things just went downhill from there ). Regardless here is a teaser of the final segment of the series. BTW, if you guys actually cared you woulda hounded me to finish. Still, I promise I’ll finish up this segment this week.

Now back into the role of glibc. The role of glibc was defined well before mature mandatory access controls like SELinux came into the Linux picture. It was defined in a general fashion that allowed it to be extended by a few userland modifications and of course the kernel support discussed in the last article.

__libc_enable_secure in rtld.c (responsible for most of the runtime linking environment cleansing)

-How does SELinux solve this problem?
-I will delve into the details… security_bprm_secureexec in binfmt_elf.c
-AT_SECURE actually enforced by glibc elf/dl-support.c and elf/dl-sysdep.c
-Static apps have no problems (aside from the usual problems)
-The linker is the only _real_ threat for dynamically linked apps.
-OK so the people that implemented SELinux were smart so we’re safe right?
-Delve into problems with shell scripts, interpreted code, etc

Continue reading »]]> Fedora 7 has been officially released. Hurry and get yours while they last. I expect Brickwall support (read -> free SELinux tools) to be out momentarily. Hmm…. I’ll go ahead and change this to FC 8 just to keep ahead of the trend

Anyways the Fedora Core 7 (FC7) release of Brickwall can be found here when available.

Continue reading »]]> Two blogs ago I felt the urge to promote something my team was working on at Tresys that was only available for Red Hat Enterprise Linux 4. Well I’m pleased to announce Tresys has released a new version of our Brickwall Security Suite for Fedora Core 6. Not only our standard version, but our professional version as well!

The professional version adds additional SELinux targets, aka protects additional services and daemons, and adds the ability to create a custom policy with little to no knowledge of SELinux.

I highly suggest Fedora users start checking out Brickwall as opposed to listening to most bloggers that tell you to just “disable SELinux by setting the Grub command line to selinux=0″. This is a far superior solution since you have all the security benefits of SELinux with an amazing amount of configurability and ease-of-use.

Check it out.

Continue reading »]]> Been awhile but been busy.

The company I work for just released our first product. It’s an application that makes using SELinux much easier. There are three versions: standard, professional, and enterprise. I think enterprise is pretty damn cool. It allows you to remotely manage SELinux policy for groups of machines from a single location. So you can easily configure different security policies on your payroll machines, your student labs, your network admin boxes, and your web servers and mail servers.

Remember, security doesn’t have to be hard. Try Tresys Brickwall today!

</shameless promotion>

Continue reading »]]> I realized after posting the first article in this series that I gave very little indication on where I plan to head with this subject. So perhaps a game plan is an order.

Errata: At the beginning I planned to focus on the environment and SELinux. I hoped to explain how certain types of applications still remain vulnerable to environmental contamination even in the face of MAC like SELinux. This would simply involve a discussion of the environment, the noatsecure permission (or lack thereof), and those environmental factors which are handled by this permission and those that are still capable of influencing execution. However, after starting the article I realized that such an explanation would stop short, probably being of little value to readers in the end and leaving me feeling slightly unsatisfied.

We started in the first article by exploring the environment, the linker, and the names of the environment variables that can influence the execution. In this article we will be discussing the kernel code that facilitates enforcement of some level of environmental protection in userland. After discussing the kernel the next article will jump back into the userland and glibc and delve into the similarities between suid/sgid protection and the SELinux noatsecure permission. Finally I would like to wrap up with what started all of this, a discussion about environment vulnerabilities that still exist even in the presence of strong MAC like SELinux; this will focus on scripting and interperted languages. I apologize for the administrative errata but I wanted to layout a road map. Now, onto the kernel discussion…

Why are we talking about the kernel here?
Well I could just come out and tell you why things are the way they are, and this is what the 3rd or 4th part in this series will do. But this is not the point of my articles in general. I like exploring the whys behind things and I take my articles down that path with me. If you don’t like knowing the why you could probably just skip this article and perhaps the next one… the end result won’t be discussed until the last article in this series.

Last time I alluded to the role glibc plays in securing the environment. I said that the secret to securing the environment lay in the appropriately named __libc_enable_secure within glibc. But before we explore glibc any further we should briefly visit the kernel.

Well that took awhile but I think we’re ready to get back on track. So we have this security_ops structure with a bunch of function pointers within. For this series all we’re really worried about is a single function pointer, bprm_secureexec. From include/security.h[1]:

* @bprm_secureexec:
* Return a boolean value (0 or 1) indicating whether a "secure exec"
* is required. The flag is passed in the auxiliary table
* on the initial stack to the ELF interpreter to indicate whether libc
* should enable secure mode.
* @bprm contains the linux_binprm structure.


If traditional capabilities are used or if CONFIG_SECURITY is enabled but no specific module is loaded (eg the “dummy” module is in use) than the traditional capability hooks perform two checks:

1. user identifier (uid) is not equal to effective user identifier(euid)
2. group identifier (gid) is not equal to effective group identifier (egid)

If either of these cases is true than the bprm_secureexec will return true.

So this is fine and dandy for the capabilities and the dummy module, but what happens with security modules? Well security models that need to protect their definition of a target process from their definition of a source also render a decision in a similar fashion. SELinux renders a decision based on the domains involved. If a process execv()s the SELinux security hook comparies the two SIDs, the original SID of the parent process and the SID of the new child process (the in-core integer representation of the home readable security context). If the two SIDS are different it means a domain transition occured. It is under these circumstances that SELinux must protect the child process from environmental influence from the parent process. The selinux_bprm_secureexec function found in security/selinux/hooks.c on line 1575 is responsible for rendering that decision from SELinux.

Capabilities, LSM/SELinux, etc, all render a decision but what happens with the result of this decision? We only need to look in two files to have a basic understanding of what happens here: include/linux/auxvec.h and fs/binfmt_elf.c. Lets start with auxvec.h first. We really only need it for a single piece of information but the usage of that information may lead to confusion. If you open that file and read the first comment you will see that the file contains “Symbolic values for the entries in the auxiliary table put on the initial stack”. Huh? Basically it defines a set of names and their associated values. The names/values are used in when constructing an executable in memory. These values are really just indices into a table put on the stack of an executable program by the ELF loader in the kernel. If you’re still feeling a little woozy these numbers are basically used as an index to store a value. That single piece of information I mentioned above can be found on line 27, the definition of AT_SECURE. The value assigned to AT_SECURE is 23… remember that… twenty-three.

So now we can move into fs/binfmt_elf.c. We’re looking for the AT_SECURE again, it’s only in one place, on line 207:
NEW_AUX_ENT(AT_SECURE, (elf_addr_t) security_bprm_secureexec(bprm));

Huh? Well there is a macro defined a few lines up that might help. If we use the expand that NEW_AUX_ENT macro we get this:
do { elf_info[ei_index++] = AT_SECURE; elf_info[ei_index++] = security_bprm_secureexec(bprm); } while (0)

Beginning to make more sense? Good. In case it doesn’t just know that elf_info stores what are basically name-value pairs in alternating locations in the array. The name, which was #defined to a value above, comes first followed by a value. So we’re storing the decision made by the security server regarding secure execution (aka cleaning the environment) at a location immediately following the location that contains AT_SECURE (the number 23).

That’s it for now… if you forget everything else by the time the next article is posted please remember one thing: the decision made in the bprm_secureexec function, regardless of the security model (basic capabilities or SELinux or another LSM), is stored in the ELF table contain interpretor information at the index indentified by AT_SECURE…. ah forget it. I can’t even remember all of that, just remember 23.

[0] AT_SECURE kernel patch post

[1] All research was based on kernel 2.6.15 w/ Gentoo patchset r7.

Continue reading »]]> While working on the second article in the environmental contamination series I found that 1/2 of the article was spent wading through the security structure and Flask implementation in the kernel. Since this is an important and recurring topic I figured I would split it out into a separate article and just link to this from future articles. I want to warn you, as most shallow explanations of kernel mechanisms, this may seem confusing since some details are left out. I encourage you to explore these on your own until you have satisfied your craving.

When it comes to making certain security decisions the kernel makes use of capabilities or other mechanisms (such as SELinux) which render a decision that the kernel then enforces or passes off to userland for use there. The kernel asks for these decisions through function calls or “hooks”. We need to have a rudimentary understanding of how these hooks work if we’re to understand how the security mechanism behind SELinux.

First place we need to look is include/security/security.h[2]. If you’re drawn into the “why”s I suggest you backtrace from the SELinux hooks (security/selinux/hooks.c) to the security operations structure (include/linux/security.h) but we will go straight to security.h. Look at the *_bprm_secureexec function headers. This highlights that how the security hooks and capabilities work with and without of security modules. Look at the #ifdef on lines 91 and the corresponding #else on line 1949. If the CONFIG_SECURITY kernel config option IS NOT enabled this security check falls back to the traditional capability security checks[1]. If the CONFIG_SECURITY option IS enabled this check propagates through one or more of several security hooks.

The kernel uses a security struct provided by LSM called security_ops (line 1037) to support a variety of security models. The security_ops maintains a set of pointers to functions that are defined by the security model you use. Some of the pointers are used during initialization, others shutdown, and others are the actual security hooks described above. These security hooks are called from various critical points in the kernel deemed to be security relevant[3]. The kernel LSM is a partial implementation of the Flask architecture with some caveats such as revocation and caching[4]. Thus, the security hooks simply return a decision regarding the access, a simple yes or no, and the place that called the hook is responsible for enforcing this decision[5].

As a quick example lets look at the ability to set our user id. If we look in kernel/sys.c we will find the function sys_setuid on line 779. The first thing done in this function (aside from delcaring variables) is to check with the security_task_setuid function (line 785). This function is defined in include/linux/security.h on lines 2346 and 1714. Both are very simple inline functions. The former is used if CONFIG_SECURITY is not enabled and merely returns 0. The latter is used when CONFIG_SECURITY is enabled and simply calls the security hook found in the security_ops task_setuid function pointer. So here we see that separation of the decision from the enforcement as defined by the Flask architecture. The decision is rendered by the hook, or security server, and the decision is enforced by caller, or the object manager.

If CONFIG_SECURITY is enabled but no specific security module is installed it means no real security_ops struct is available so it falls through the dummy security_ops which just passes it onto the capability hooks mentioned above. However, if a security module is installed it leverages these hooks instead of the “dummy” hooks.

Hopefully this is enough to help one wade through the kernel enforcement and security server mechanism.

[1] As far as I know, secureexec was not part of the POSIX capability specification, rather it is a Linux specific extenstion to this specification. Please correct me if I’m wrong on this point (and all other points ).

[2] The research for this article was done on the 2.6.15 kernel with the Gentoo r7 patchset applied.

[3] “Security relevant” is arguable as is the acutal functionality of the LSM. If you follow the LSM and LKML you will find that the LSM implementation is sub-optimal for most security models and has it’s flaws. None the less this is the best model we have at the moment so we have to make the best of it.

[4] Some of these issues, such as caching, are addressed by the SELinux module itself, but not by LSM.

[5] Refer to this excerpt from the Flask paper for a diagram depicting this separation of enforcement from decision.