Wednesday, November 21, 2012

Debugging Shell Scripts in Unix

Debugging is an important part of your overall development process. Unfortunately not many developers know that even shell scripts can be debugged. The good thing is debugging shell scripts is just about knowing a few options of the 'sh' command. Lets see how we can debug the shell scripts.

To see exactly where the script produces an error use the 'sh' command with a 'x' option:

sh -x myscript.sh your-arguments-if-you-hahve

When you run the above command you can see each statement with their output printed in the next line.

For example if i have a script which does addition of two numbers given by the user like:

#!/bin/sh
echo $1
echo $2
echo $(($1 + $2))

The output of 'sh -x myscript.sh 3 4' would be

+ echo 3
3
+ echo 4
4
+ echo 7
7

So, we see the statement and its output in the next line.

Other options are:

-e in non-interactive mode, exit immediately if a command fails.

-v print shell input lines as they are read.

-n read commands but do not execute them.

In the other options 'v' is probably a really good option to explore. If i run my above script with '-v' option then the following output would be seen:

sh -v myscript.sh 3 4

#!/bin/sh
echo $1
3
echo $2
4
echo $(($1 + $2))
7

This is again a good option to explore.


Posted by at 1 comment:
Labels: , , ,

Thursday, September 13, 2012

Recursive Automatic download of FTP files in Unix

With unix there comes numerous built in beautiful utilities. If one understands these utilities fully, he/she can accomplish seemingly difficult tasks in a jiffy.

One such thing that we always try to do it downloading a bunch of files from a FTP site. This becomes even tricky if there are many sub folders.

One such utility is wget.

A command line to recursively download all the files from an FTP location:

wget -r ftp://user:password@myremotehost -o /tmp/log &

'&' will make the process background, and output in /tmp/log file.

Easy money.


Posted by at No comments:

Sunday, August 5, 2012

Though personally I am not too much in favor of singletons but its a good brain exercise to see different ways in which it could be implemented.

The standard way as we all know is you have a final class (ideally purely for performance optimizations) with a private constructor and a public static getter method. I personally like to create the singleton object eagerly. It saves all the hassles to keep the getter method synchronized and therefore, make it a lot more scalar.

With the advent of generics, their application with different data structures and design patterns evolved and so did a couple of variants of Singleton.

class GenericSingleton {

private static final GenericSingleton instance = new GenericSingleton();
private GenericSingleton(){ }
public static GenericSingleton getInstance()
{
return (instance);
}
}

There is one more way to implement this and that being creating a map of identifiers and their classes but personally i prefer the above way as i prefer casting being kept out of the singleton itself.

Posted by at No comments:

Wednesday, June 27, 2012

One obviously knows a lot of different methods to implement singleton in java. Singleton by definition means one object per classloader. Starting Java 5 we now have enums which are first class singleton objects. The best method in java for implementing singletons is via enum(s).

Lets take a look with an example of a DB connection pool manager. We're using Data source to get connections:

public enum DBConnectionManager {
INSTANCE;

private DataSource ds = null;
private Lock connectionLock = new ReentrantLock();

DBConnectionManager() {
try {
final Context initCtx = new InitialContext();
final Context envCtx = (Context) initCtx.lookup("java:comp/env");
ds = (DataSource) envCtx.lookup("jdbc/DBConnectionManager");
} catch (NamingException e) {
e.printStackTrace();
}
}

public Connection getConnection() throws SQLException {
if(ds == null) return null;

Connection conn = null;
connectionLock.lock();
try {
conn = ds.getConnection();
} finally {
connectionLock.unlock();
}

return conn;
}
}


Since we are using connection pooling, we do not really need to close the connection and it's the responsibility of the calling code to close the connection (which actually does nothing but returns the connection back to the pool).

Posted by at No comments:

Wednesday, June 6, 2012



Get a list of all the running Threads in the current JVM and its origination


Sometimes, it makes sense to know the list of all running threads in the JVM. Either when you are creating your own profiling tool, or, when you need to monitor the health of your system and need meta-information of your platform. You may also need the information of all running threads when you would like to selectively, say 'stop' the threads (a technique i would showcase in my subsequent posts).


There are two ways to get it:

1. The long way:


Get the root thread group. and then call the enumerate() function on the root group repeatedly.


ThreadGroup rootGroup = Thread.currentThread( ).getThreadGroup( );
ThreadGroup parentGroup;
while ( ( parentGroup = rootGroup.getParent() ) != null ) {
rootGroup = parentGroup;
}



Thread[] threads = new Thread[ rootGroup.activeCount() ];
while ( rootGroup.enumerate( threads, true ) == threads.length ) {
threads = new Thread[ threads.length * 2 ];
}

for(Thread t : threads)
{
System.out.println("Thread name:"+t.getName());
StackTraceElement[] stackTraceElements = t.getStackTrace();
for(StackTraceElement s: stackTraceElements)
{
System.out.println("class name="+s.getClassName()+" and method name="+s.getMethodName()+" at line no.="+s.getLineNumber() );
}
}


2. The short way:


Set threadSet = Thread.getAllStackTraces().keySet();
//now just for displaying get the threads
threads = threadSet.toArray(new Thread[threadSet.size()]);
for(Thread t : threads)
{
System.out.println("Thread name:"+t.getName());
StackTraceElement[] stackTraceElements = t.getStackTrace();
for(StackTraceElement s: stackTraceElements)
{
System.out.println("class name="+s.getClassName()+" and method name="+s.getMethodName()+" at line no.="+s.getLineNumber() );
}
}




Posted by at No comments:
Labels: ,

Tuesday, May 22, 2012

Resolving "too many connections" error in MySQL/Java.

Resolving "too many connections" error in MySQL/Java.

First Solution:

If you are getting "too many connections" errors in MySQL you can could resolve this by changing the max_connections setting to allow more connections.
The default setting for max_connections is 151. You can see what the current setting is by running the following SQL command from the MySQL command line or MySQL query browser or any similar tool:

show variables like "%max_connections%";


This will return the following ResultSet:

+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+

As visible above my current max connections settings is 151, which means at any given point in time i can have max 151 simultaneous connections.

You can change the setting to e.g. 250 by issuing the following command without having to restart the MySQL server :

set global max_connections = 250;


This will take effect immediately, but will be forgotten the next time MySQL is restarted. To make the change permanent you need to edit the my.cnf configuration file. On Ubuntu, Red Hat and similar distributions this is at /etc/my.cnf; other distros will store it elsewhere.

Under the [mysqld] section add the following setting:

max_connections = 250


The above line would make default connections limit to 250 for MySQL.

Note:

You can set the max_connections limit only if you have sufficient MySQL access rights.The max_connections limit includes any sleep(ing) connections on the mysql side. Furthermore, the maximum number of connections MySQL can support depends on the quality of the thread library on a given platform, the amount of RAM available, how much RAM is used for each connection, the workload from each connection, and the desired response time. Linux or Solaris should be able to support at 500 to 1000 simultaneous connections routinely and as many as 10,000 connections if you have many gigabytes of RAM available and the workload from each is low or the response time target undemanding. Windows is limited to (open tables × 2 + open connections) < 2048 due to the Posix compatibility layer used on that platform. Also sometime it may be necessary to increase open-files-limit system variable's limit.

Other solutions:

If the above solution doesn't resolve the problem, then it has be to one of the below mentioned issues:

1. Disk full:

Yes, if your disk is full, then also you would get this error. To resolve this please remove any unnecessary log files, or any other files that could be using disk space unnecessarily.

2. RAM full:

If you RAM is full, assuming some process like a badly coded java application is hogging all the RAM then also you would get "too many connections" error for mysql. To resolve this kill the process that's taking your RAM unnecessarily and re-start the MySQL service (you my try it without restarting the service but it may rarely work).

The last two solutions are tricky to diagnose as common wisdom would suggest just increase the max_connections should do the trick, but its not always the case, speaking from my own experience.
Posted by at 1 comment:
Labels: , ,
Subscribe to: Posts (Atom)