Peter Bieringer2013-10-26 14:23:17 UTCDescription of problem:While Apache 2.0 (CentOS 5) and 2.2 (CentOS 6) have no issues with chained pipes in CustomLog directive, it looks like that 2.4.6 from Fedora 19 won't support this anymore.Version-Release number of selected component (if applicable):httpd-2.4.6-2.fc19.i686How reproducible:AlwaysSteps to Reproduce:1. Create additional CustomLog directives with (chained) pipes, e.g.# Anonymized logCustomLog ' /usr/local/bin/ipv6loganon -f -a /var/log/httpd/access-anonlog' combined# Default log with cronolog extensionCustomLog ' /usr/sbin/cronolog /var/log/httpd/access.log-%Y%m%d' combined# Anonymized log with cronolog extensionCustomLog ' /usr/local/bin/ipv6loganon -f /usr/sbin/cronolog /var/log/httpd/access-anon.log-%Y%m%d' combined2.
Start Apache3. Submit a curl requestActual results:# ll /var/log/httpd/access.-rw-r-r-.
Piping in Unix or Linux A pipe is a form of redirection (transfer of standard output to some other destination) that is used in Linux and other Unix-like operating systems to send the output of one command/program/process to another command/program/process for further processing.
1 root root 81 Oct 26 16:10 /var/log/httpd/access-anonlog-rw-r-r-. 1 root root 81 Oct 26 16:10 /var/log/httpd/accesslog-rw-r-r-.
The Complete Guide Table of Content.CFEngine is a configuration management system that provides a framework for automated management of IT infrastructure.CFEngine is decentralized and highly scalable. $ /var/cfengine/bin/cf-agent -bootstrap public IP of localhostThen execute the same step (using the exact same IP)on all hosts that should pull policy from that server.CFEngine will create keys if there are none present, and exchangethose to establish trust.CFEngine will output diagnostic information upon bootstrap. In case of error,investigate the promises the server is making (run inverbose mode on the policy hub for more informative messages). Note thatby default, CFEngine's server daemon trusts incoming connectionsfrom hosts within the same /16 subnet.After a host has been bootstrapped, the text file policyserver.dat inthe CFEngine installation contains the IP address of the policy server. Key exchangeThe key exchange model used by CFEngine is based on that used by OpenSSH. Itis a peer to peer exchange model, not a central certificate authority model.This means that there are no scalability bottlenecks (at least by design,though you might introduce your own if you go for an overly centralizedarchitecture).Key exchange is handled automatically by CFEngine and all you need to do is todecide which keys to trust.
The server trusts new keys onlyfrom addresses in. Once a key has beenaccepted you should close down list. Then, even if a malicious peeris spoofing an allowed IP address, its unknown key will be denied.Once you have arranged for the right to connect to the server, you must decidewhich hosts will have access to which files. This is done with promises.
$ WARNING - You do not have a public key from host ubik.iu.hio.no = 128.39.74.25$ Do you want to accept one on trust? (yes/no)-Once public keys have been exchanged from client to server and from server toclient, the issue of trust is solved according to public key authenticationschemes.
You only need to worry about trust when one side of a connection hasnever seen the other side before. Time windows (races)All security is based on a moment of trust that is granted by a user at somepoint in time – and is assumed thereafter (once given, hard to rescind).Cryptographic key methods only remove the need for a repeat of the trustdecision.
After the first exchange, trust is no longer needed, because thekeys allow identity to be actually verified.Even if you leave the trust options switched on, you are not blindly trustingthe hosts you know about. The only potential insecurity lies in any new keysthat you have not thought about. If you use wildcards or IP prefixes in thetrust rules, then other hosts might be able to spoof their way in on trustbecause you have left open a hole for them to exploit. That is why it isrecommended to set the system to the state of zero trustimmediately after key transfer, by commenting or emptying out the trust options( on the server).It is possible, though somewhat laborious, to transfer the keys out of band,by copying /var/cfengine/ppkeys/localhost.pub to/var/cfengine/ppkeys/user-aaa.bbb.ccc.mmm (assuming IPv4) on another host.e.g. Localhost.pub - root-128.39.74.71.pubOther users than rootCFEngine normally runs as user 'root' (except on Windows which doesnot normally have a root user), i.e. A privileged administrator.
Ifother users are to be granted access to the system, they must alsogenerate a key and go through the same process. In addition, theusers must be added to the server configuration file. EncryptionCFEngine has 2 communication protocols. Classic or 1 and 2 or latest.Each protocol provides different encryption options for keeping file contentsprivate during transfer.However, the main role of encryption in configuration management is forauthentication. Secrets should not be transferred through policy, encrypted ornot.
Policy files should be considered public, and any leakage should notreveal secret information.Note: Connections from the cf-agent are cached as described in thedocumentation for body. Protocol ClassicEncryption for Enterprise is symmetric AES 256 bit in CBC mode, usinga session key exchanged during the RSA handshake.In core/community as outgoing outlined in thedocumentation the initialconnection is encrypted using the public/private keys for the clientand server hosts. After the initial connection is establishedsubsequent connections and data transfer is encrypted by a randomlygenerated Blowfish key that is refreshed each session.With the classic protocol cf-serverd has the ability to enforce that afile transfer be encrypted by setting the. When ACLs thatrequire encryption have unencrypted access attempts cf-serverd logs anerror message indicating the file requires encryption.
Access to filesthat cf-serverd requires to be encrypted can be logged by setting the. Protocol 23.6 introduced a new protocol option for communication withcf-serverd.is the default in 3.7+ and uses a TLS session for encryption.Note: When protocol 2 is in use legacy encryption attributes are noop.The following attributes are affected:. in copy from bodies.
in in access promises. in body commoncontrolThe specific encryption algorithm used depends on the ciphernegotiated between the client and the server.
You can control whichciphers are allowed by cf-serverd for incoming connections bysetting the. Controllingwhich ciphers are allowed to be used in outgoing connections isdone by setting.Additionally the minimum version of TLS required for incomingconnections can be set inand the minimum version of TLS required for outgoing connectionscan be set in.There are debug and verbose level logs produced by cf-agent toindicate when TLS is in use.The following was captured by running the agent update policy in debugmode./var/cfenigne/bin/cf-agent -Kdf update.cf. PlatformVersionsArchitecturesAIX7.1, 7.2PowerPCCentOS/RHEL5, 6, 7x86-64, x86Debian7, 8, 9x86-64, x86HP-UX11.31+ItaniumSLES11x86-64, x86Solaris11UltraSparcSolaris10UltraSparc, x86Ubuntu14.04, 16.04, 18.04x86-64, x86Windows2008x86-64, x86Windows2008, 2012x86-64also includes platform-specific notes.CFEngine Enterprise has from IBM.This means that CFEngine Enterprise has been technically verified by IBMto be installed in and manage VIOS environments. Future platform supportThe CFEngine team will continue to support future releases of popular Hostplatforms, including RHEL, Debian, Ubuntu, as well as maintaining support forexisting platforms important to users.In general, CFEngine is known to run on a wide range of other platforms.
As longas the platform is POSIX compliant and has a C compiler toolchain that fullyimplements the C99 standard, we are happy to work with you to make CFEngineavailable. Please for details.Known IssuesCFEngine defects are managed in our.
Please reportbugs or unexpected behavior there, following the documented guideline for newbug reports. Core Issues affectingThe items below highlight issues that require additional awareness when startingwith CFEngine or when upgrading from a previous version. Cf-agent -N or cf-agent -negate is not workingAs reported in thefunctionality of negating persistent classes on the command line, wasremoved sometime before 3.5, commitcf63dbcaa5bfd5d12b21. The ticket is open until thefunctionality is reinstated. HP-UX specific. do not have out-of-the-box support for the HP-UXspecific package manager.
The workaround is to call the package managerdirectly using. Some important system information is missing from the HP-UX inventory report,as well as from CFEngine hard classes and system variables. The workaround isto use system tools to obtain the required information and set classes basedon this. Disk free. Memory size.
Several OS and architecture specific attributes. System version. System serial number. System manufacturer. CPU model.
BIOS version. BIOS vendor. depend on the ps native tool, which bydefault truncates lines at 128 columns on HP-UX.
It is recommended to editthe file /etc/default/ps and increase the DEFAULTCMDLINEWIDTH settingto 1024 to guarantee that process promises will work smoothly on theplatform. Upgrading CFEngine on HP-UX is not supported by the out-of-the-box policy.There is awith a workaround.Enterprise monitoring graphsMonitoring graphs are disabled by default in CFEngine Enterprise 3.6 and laterversions. To enable them, change monitoringinclude inmasterfiles/controls/reports.cf to e.g. Note that this can havesignificant impact on the resource consumption of your hub.Monitoring graphs are not supported on all platforms, currently AIX, HP-UX andWindows do not have this data. Enterprise inventory CSV report is empty (0 bytes)Exporting a CSV-based inventory report can result in a 0-byte length fileif the CFEngine Server is accessed over https and the certificate's CNmismatches with the URL you use to export the report. To verify this is theproblem, check the Mission Portal application logs (currently at/var/cfengine/httpd/htdocs/application/logs) on the CFEngine Server.
If yousee lines like the following you affected by this issue. RedHat/CentOS/SUSE $ rpm -i.rpmDebian/Ubuntu $ dpkg -i.debNote: Install actions logged to /var/logs/cfengine-install.log. BootstrapBootstrapping a client means to configure it initially. With CFEngine, the default bootstrap:.
records the server's address (accessible as ) and public key, and gives the server the client's key to establish trust (see ). copies all the contents of /var/cfengine/masterfiles on the policy server (AKA ) to /var/cfengine/inputs (AKA ). See for details.Run the bootstrap command, first on the policy server:.Find the IP address of your Policy Server. $ sudo /var/cfengine/bin/cf-agent -bootstrap The bootstrap command must then be run on any client attaching itself to this server, using the ip address of the policy server (i.e. Exactly the same as the command run on the policy server itself). Post-Installation ConfigurationCFEngine itself is configured through policy as well (see andfor details). The following basic changes to the default policy will configureand for your environment.
Configure agent email settingsBy default an email a summary of any run initiated. Youmay want to adjust the mailto or mailfrom. If you have a centralized reportingsystem like CFEngine Enterprise you may wish to disable agent emails alltogether. Configure mailto and mailfromThe preferred way of setting def.mailfrom is from the. $ wget && sudo bash./quick-install-cfengine-enterprise.sh hubThis script installs the latest CFEngine Enterprise Policy Server on your server machine. Bootstrap the Policy Server.
The Policy Server must be bootstrapped to itself. Find the IP address of your Policy Server:$ ifconfig.Run the bootstrap command: sudo /var/cfengine/bin/cf-agent -bootstrap Example: $ sudo /var/cfengine/bin/cf-agent -bootstrap 172.31.3.25Upon successful completion, a confirmation message appears: 'Bootstrap to '172.31.3.25' completed successfully!' .Type the following to check which version of CFEngine your are running:/var/cfengine/bin/cf-promises -version.The Policy Server is now installed.Step 3. Install Enterprise on Host (Client). Ensure you are logged into the host machine setup earlier. Install CFEngine client version using the following. $ wget && sudo bash./quick-install-cfengine-enterprise.sh agentNote: The installation will work on 64-bit and 32-bit client machines (the host requires a 64-bit machine).The client software (host), has been installed on the second virtual machine.Note: You can install CFEngine Enterprise on up to 25 hosts using the script above.
Bootstrap the Host to the Policy Server. All hosts must be bootstrapped to the Policy Server in order to establish a connection between the Host and the Policy Server.Run the same commands that you ran in Step 2, $ sudo /var/cfengine/bin/cfagent bootstrap.Example: $ sudo /var/cfengine/bin/cfagent bootstrap 172.31.3.25.The installation process is complete and CFEngine Enterprise is up and running on your system.Step 5. Log in to the Mission Portal. The Mission Portal is immediately accessible.
Connect to the Policy Server through your web browser at: (Note: The External IP address is available in the AWS console). The default username for the Mission Portal is admin, and the password is also admin. The Mission Portal runs TCP port 80 by default. During the initial setup, the Host(s) might take a few minutes to show up in the Mission Portal. Refresh the web page and login again if necessary.What Next?
Tutorials.Whereas the first tutorial in this list teaches you how to deploy business policythrough the Mission Portal, this advanced, command-line tutorial shows you how to distribute policy files from the Policy Server to all pertinent Hosts.Recommended Reading.Installing Enterprise 25 FreeThese instructions describe how to install the latest version of CFEngine Enterprise 25 Free. This is the fullversion of CFEngine Enterprise, but the number of Hosts (clients) is limited to 25.Note the following requirements:. To install this version of CFEngine Enterprise, your machine must be running a recent version of Linux.This installation script has been tested on RHEL 5 and 6, SLES 11, CentOS 5 and 6, and Debian 6 and 7. You need a minimum of 2 GB of available memory and a modern 64 bit processor. Plan for approximately 100MB of disk space per host. You should provide anextra 2G to 4G of disk space if you plan to bootstrap more hosts later.
You need a least two VMs/servers, one for the Policy Server and one for a Host (client). They must be on the same network. The Policy Server needs to run on a dedicated OS with a vanilla installation (i.e.
It only has repositories and packages officiallysupported by the OS vendor)Installation OverviewDuring the course of the instructions outlined in this guide, you will perform the following tasks:. Install CFEngine Enterprise onto a Policy Server and onto Hosts.A Policy Server (hub) is a CFEngine instance that contains promises (business policy) that get deployed to Hosts.Hosts are clients that retrieve and execute promises. Bootstrap the Policy Server to itself and then bootstrap each of the Hosts to the Policy Server. Bootstrapping establishes a trust relationship between the Policy Serverand all Hosts. Thus, business policy that you create in the Policy Server can be deployed to Hosts throughout your company.Bootstrapping completes the installation process. Log in to the Mission Portal. The Mission Portal is a graphical user interface that allows you to verify thethe actual state of all your Hosts, thus ensuring that your promises are being executed.
Try out the Tutorials. Links to three tutorials give you a head start on learning CFEngine.1. Download and install Enterprise on a Policy ServerPlease Note: Internet access is required from the host if you wish to use the quick install script.Run the following script on your designated Policy Server (hub) 64-bit machine (32-bit is not supported on the Policy Server). $ sudo /var/cfengine/bin/cf-agent -bootstrap Example: $ sudo /var/cfengine/bin/cf-agent -bootstrap 192.168.1.12The installation process is complete and CFEngine Enterprise is up and running on your system. Log in to the Mission PortalThe Mission Portal is immediately accessible. Connect to the Policy Serverthrough your web browser at:username: adminpassword: adminThe Mission Portal runs TCP port 80 by default.
(Clickto configure the Mission Portal to use HTTPS instead of HTTP.) During the initial setup, the Host(s) might take a few minutes to show up in the Mission Portal. Simply refresh the web pageand login again if necessary.Note: If you are running Enterprise with Vagrant, you must add thecorrect port: in your browser.
The is the port-forwardernumber you use in your Vagrantfile (e.g. Policyserver.vm.network 'forwardedport', guest: 80, host: 8080; the port will be 8080).Tutorials.Whereas the first tutorial in this list teaches you how to deploy business policythrough the Mission Portal, this advanced, command-line tutorial shows you how to distribute policy files from the Policy Server to all pertinent Hosts.Recommended Reading.Using VagrantThe CFEngine Enterprise Vagrant Environment provides an easy way to test andexplore CFEngine Enterprise.
This guide describes how to set up a client-servermodel with CFEngine and, through policy, manage both machines. Vagrant willcreate one VirtualBox VM to be the Policy Server (server), and another machinethat will be the Host Agent (client), or host that can be managed by CFEngine.Both will will run CentOS 6.5 64-bit and communicate on a host-only network.Apart from a one-time download of Vagrant and VirtualBox, this setup requiresjust one command and takes between 5 and 15 minutes to complete (determined byyour Internet connection and disk speed).
Upon completion, you are ready tostart working with CFEngine. Requirements. 2G disk space. 1G memory. CPU with VT extensions capable of running 64bit guestsNote: VirtualBox requires that your computer support hardware virtualizationin order to make use of the CentOS 64-bit virtual machines mentioned above.This is sometimes turned on or off in BIOS settings, but not all processorsand motherboards necessarily support hardware virtualization.If your system lacks this support you will need to choose another computer totake advantage of the 64-bit virtual machines. Overview. Install Vagrant.
Install Virtualbox. Start the CFEngine Enterprise Vagrant Environment. Log in to the Mission Portal. Stop CFEngine Enterprise.
UninstallInstall VagrantThis tutorial uses Vagrant to configure your VMs. It is available for Linux,Windows and MacOS and can be downloaded from vagrantup.com (this guide has beentested with version ).
Afterdownloading Vagrant, install it on your computer. You may want to reference theWindows Mac or Linux vagrant install guides. Install VirtualboxThis tutorial uses VirtualBox to create virtual machines on your computer, towhich Vagrant deploys CFEngine. VirtualBox can be downloaded fromvirtualbox.org (this guide has been tested with version). Afterdownloading VirtualBox, install it on your computer.Note: To avoid problems, disable other virtualization environments you arerunning. Start the CFEngine Enterprise 3.12 Vagrant EnvironmentStep 1. Download our ready-made Vagrant project.Step 2.
Save and unpack the file anywhere on your drive; thiscreates a Vagrant Project directory.Step 3. Open a terminal and navigate to the Vagrant Project directory (e.g./home/user/CFEngineEnterprisevagrantquickstart-3.12.2-2, or C:CFEngineEnterprisevagrantquickstart-3.12.2-2) and enter the following command. $ vagrant upVagrant performs the following processes:. Downloads the CentOS basebox used for both the hub and the client (if it hasnot already been cached by vagrant. Provisions, installs and bootstraps the hub.
Provisions, installs and bootstraps clientsThe basebox is 500MB.Note: If you want to use more hosts in this environment, you canedit the Vagrantfile text file in the directory that you have just created.Change the line that says 'hosts = 1' to the number of hosts that you want inthe setup. The maximum supported in this evaluation version of CFEngine is 25. Log in to the Mission PortalAt the end of the setup process, you can use your browser to log in to theMission Portal:adminpassword: adminNote: It may take up to 15 minutes before the hosts register in MissionPortal.That's all there is to it, the install is complete! Move on and explore the environment.
Exploring the Environment Accessing VMs Accessing via SSHThe standard vagrant ssh key is configured. To ssh to a host run vagrant sshmyhost where myhost is the name of a running vm as seen in the vagrantstatus output. Both the 'root' and 'vagrant' users passwords are set to'vagrant'.Example. $ vagrant ssh hubLast login: Fri Jun 13 18: from 10.0.2.2 Accessing via GUIIf you launch the virtualbox GUI you should find the vagrant vms namedCFEngine Enterprise 3.12.2-2 hub, and CFEngine Enterprise 3.12.2-2 agent host001. Additionally, you can uncomment the v.gui=trueoption in the Vagrantfile to have the console gui start with the vms.Note: There are two v.gui settings to uncomment; one for the hub, and onefor the clients.
Check the status of the vmsRunning vagrant status from the vagrant project directroy will produceoutput like this. $ vagrant destroyhost001: Are you sure you want to destroy the 'host001' VM? y/N y host001: Forcing shutdown of VM. host001: Destroying VM and associated drives.
host001: Running cleanup tasks for 'shell' provisioner. host001: Running cleanup tasks for 'shell' provisioner. host001: Running cleanup tasks for 'shell' provisioner. Hub: Are you sure you want to destroy the 'hub' VM?
y/N y hub: Forcing shutdown of VM. hub: Destroying VM and associated drives. hub: Running cleanup tasks for 'shell' provisioner. hub: Running cleanup tasks for 'shell' provisioner. hub: Running cleanup tasks for 'shell' provisioner. Uninstall Vagrant EnvironmentWhen you have completed your evaluation are ready to use CFEngine onproduction servers, remove the VMs that you created above by following thesesimple instructions:To remove the VMs entirely, type: vagrant destroyIf you are completely done and do not anticipate using them anymore, you canalso remove the base box centos-6.5-x8664-cfengineenterprise-vagrant-45 that wasdownloaded.
You can see it by typing vagrant box list. To delete the baseboxrun vagrant box remove centos-6.5-x8664-cfengineenterprise-vagrant-45 virtualbox.Note: Running vagrant up from the vagrant project directory again willre-download this basebox.Vagrant and VirtualBox are useful general purpose programs, so you might wantto keep them around. If not, follow the standard procedures for your OS toremove these applications. Next Steps.See also.Installing Enterprise for ProductionThese instructions describe how to install the latest version ofCFEngine Enterprise in a production environment using pre-compiled rpmand deb packages for Ubuntu, Debian, Redhat, CentOS, and SUSE. General RequirementsCFEngine recommends the following:Host MemoryDuring normal operation the CFEngine processes consume about 30 MB ofresident memory (RSS) on hosts with the agent only (not acting asPolicy Server).However there might be spikes due to e.g.
Commands executed from theCFEngine policy so it is generally recommended to have at least 256 MBavailable memory in order to run the CFEngine agent software.Host diskSo that the agent is not affected by full disks it is recommended that/var/cfengine be on its own partition.On Unix-like systems, a 500 MB partition for /var/cfengine should give yousome breathing room, typical user reported sizes are in the 100-250 MB range. OnWindows systems, CFEngine consumes more space because Windows lacks support forsparse files (which are used opportunistically by lmdb).
5 G of space shouldprovide some breathing room, typical user reported sizes for C:ProgramFilesCfengine are around 1 GB. As always things vary in different environmentsit's a good idea to measure consumption in your infrastructure and customizeaccordingly.The agent builds local differential reports for promise outcomes. Thelonger the period between collections from the enterprise hub the moreresources are required to calculate these differentials.