How to trace what’s being done by user’s on your Linux Server

How to trace what’s being done by user’s on your Linux Server 

Intrusions can take place from both authorized (insiders) and unauthorized (outsiders) users. My personal experience shows that unhappy user can damage the system, especially when they have a shell access. Some users are little smart and removes history file (such as 

~/.bash_history) but you can monitor all user executed commands.

It is recommended that you log user activity using process accounting. Process accounting allows you to view every command executed by a user including CPU and memory time. With process accounting sys admin always find out which command executed at what time :)

The psacct package contains several utilities for monitoring process activities, including ac, lastcomm, accton and sa. 

The ac command displays statistics about how long users have been logged on. 

The lastcomm command displays information about previous executed commands. 

The accton command turns process accounting on or off. 

The sa command summarizes information about previously executed commmands. 

Install psacct or acct package 

Use up2date command if you are using RHEL ver 4.0 or less 

# up2date psacct 

Use yum command if you are using CentOS/Fedora Linux / RHEL 5: 

# yum install psacct 

Use apt-get command if you are using Ubuntu / Debian Linux: 

$ sudo apt-get install acct OR # apt-get install acct 

Start psacct/acct service 

By default service is started on Ubuntu / Debian Linux by creating /var/account/pacct file. But under Red Hat /Fedora Core/Cent OS you need to start psacct service manually. Type the following two commands to create /var/account/pacct file and start services: 

# chkconfig psacct on 

# /etc/init.d/psacct start

Now let us see how to utilize these utilities to monitor user commands and time. 

Display statistics about users' connect time 

ac command prints out a report of connect time in hours based on the logins/logouts. A total is also printed out. If you type ac without any argument it will display total connect time: 

[santosh@Redhat ~]$ ac 

total 53.17 

Display totals for each day rather than just one big total at the end: 

[santosh@Redhat ~]$ ac -d 

Jul 23 total 19.65 

Aug 5 total 12.10 

Aug 6 total 20.67 

Today total 0.80 

Display time totals for each user in addition to the usual everything-lumped-into-one value: 

[santosh@Redhat ~]$ ac -p 

santosh 0.09 

raj 0.97 

root 47.07 

sohan 5.11 

total 53.24 

find out information about previously executed user commands 

Use lastcomm command which print out information about previously executed commands. You can search command using usernames, tty names, or by command names itself. 

Display command executed by santosh user: 

[santosh@Redhat ~]$ lastcomm santosh 

lastcomm santosh pts/2 0.00 secs Tue Aug 7 07:18 

ac santosh pts/2 0.00 secs Tue Aug 7 07:17 

ac santosh pts/2 0.00 secs Tue Aug 7 07:16 

ac santosh pts/2 0.00 secs Tue Aug 7 07:16 

grep santosh pts/2 0.00 secs Tue Aug 7 07:11 

dircolors santosh pts/2 0.00 secs Tue Aug 7 07:11 

bash F santosh pts/2 0.00 secs Tue Aug 7 07:11 

tput santosh pts/2 0.00 secs Tue Aug 7 07:11 

tty santosh pts/2 0.00 secs Tue Aug 7 07:11 

Search the accounting logs by command name: 

[santosh@Redhat ~]$ lastcomm useradd 

useradd S root pts/1 0.10 secs Tue Aug 7 07:11 

useradd F root pts/1 0.00 secs Tue Aug 7 07:11 

useradd F root pts/1 0.00 secs Tue Aug 7 07:11 

useradd F root pts/1 0.00 secs Tue Aug 7 07:11 

useradd F root pts/1 0.00 secs Tue Aug 7 07:11 

Search the accounting logs by terminal name pts/1 

[santosh@Redhat ~]$ lastcomm pts/1 

passwd S root pts/1 0.04 secs Tue Aug 7 07:11 

useradd F root pts/1 0.00 secs Tue Aug 7 07:11 

psacct root pts/1 0.01 secs Tue Aug 7 07:11 

touch root pts/1 0.00 secs Tue Aug 7 07:11 

accton S root pts/1 0.00 secs Tue Aug 7 07:11 

Summarizes accounting information 

Use sa command to print summarizes information about previously executed commands. In addition, it condenses this data into a summary file named savacct which contains the number of times the command was called and the system resources used. The information can also be summarized on a per-user basis; sa will save this iinformation into a file named usracct. 

[santosh@Redhat ~]$ sa 

4 0.36re 0.12cp 31156k up2date 

216 1.82re 0.02cp 647k 

19 1.64re 0.02cp 642k ***other* 

2 0.01re 0.00cp 4123k rhsmd 

48 0.01re 0.00cp 649k find 

33 0.01re 0.00cp 504k tmpwatch 

2 0.01re 0.00cp 662k logrotate 

54 0.14re 0.00cp 687k awk 

14 0.00re 0.00cp 498k basename 

13 0.00re 0.00cp 495k logger 

12 0.00re 0.00cp 743k makewhatis* 

3 0.00re 0.00cp 505k tr 

3 0.00re 0.00cp 496k renice 

3 0.00re 0.00cp 507k rm 

2 0.00re 0.00cp 596k grep 

2 0.00re 0.00cp 742k prelink* 

2 0.00re 0.00cp 742k makewhatis.cron* 

2 0.00re 0.00cp 551k sed 

2 0.00re 0.00cp 495k ionice 

Take example of first line: 

4 0.36re 0.12cp 31156k up2date 

Where, 

0.36re "real time" in wall clock minutes 

0.12cp sum of system and user time in cpu minutes 

31156k cpu-time averaged core usage, in 1k units 

up2date command name 

Display output per-user: 

# sa -u 

root 0.00 cpu 595k mem accton 

root 0.00 cpu 12488k mem initlog 

root 0.00 cpu 12488k mem initlog 

root 0.00 cpu 12482k mem touch 

root 0.00 cpu 13226k mem psacct 

root 0.00 cpu 595k mem consoletype 

root 0.00 cpu 13192k mem psacct * 

root 0.00 cpu 13226k mem psacct 

root 0.00 cpu 12492k mem chkconfig 

postfix 0.02 cpu 10696k mem smtpd 

santosh 0.00 cpu 19328k mem userhelper 

santosh 0.00 cpu 13018k mem id 

santosh 0.00 cpu 13460k mem bash * 

lighttpd 0.00 cpu 48240k mem php 

* 

Display the number of processes and number of CPU minutes on a per-user basis 

[root@Redhat ~]# sa –m 

root 566 3.44re 0.22cp 991k 

postfix 2 0.21re 0.00cp 3138k