Pages

Monday, December 28, 2009

Thursday, September 24, 2009

unable to connect to MySQL Or MySql is not Allowing more Connections

some times aborting connections or interrupting process lead to another problem to server from stop giving connections to client. that is
server do not allow any more connections to client.
probable causes would be.
client is idle more than wait_timeout or
due to long queries getting aborted from client or
client makes mistake in authentications while connecting to database.
all these lead to increase in aborted_client counter of server variable.

and if max_connect_errors= (default is 10) is set to very low will cause a server to give more connection to client when it goes beyond limit
so its better to set it something more (normally 5000-10000) which will minimize this error.

please see more ref. I found for same

http://dev.mysql.com/doc/refman/5.0/en/blocked-host.html
http://dev.mysql.com/doc/refman/5.0/en/communication-errors.html
http://lists.mysql.com/internals/33476

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException

I have noticed this error in my MySql Outage, suddenly going down due to overload.
after my observation I found that this kind of error mostly will cause because of
system variables settings of wait_timeout and interactive_timeout on mysql server is set lower than datasource
idle timeout period.
my datasource pooling property for conection time out is set to 1 min
on server variable wait_timeout is set to 28 sec. (28800 milisec)

in most cases where connection is idle from host and if server wait_timeout is less than client datasource timeout
server closes connection after its sleep time is more than wait_timeout or interactive_timeout period. and client is unaware of this
and when he fires query again on same connection gets above error.

to avoid this there are following ways
1. if you are using plain connection using jconnector set autoconnect property to true.
2. if you have custom connection poolilng machanism try to validate your connection before using it from pool.
3. in other case try to set your server variable values for wait_timeout, interactive_timeout greater than your datasource timeout.

Monday, August 31, 2009

Upgrading MySQL 4.x to 5.x

This note give brief about how to uninstall MYSQL 4.1.xx and install MYSQL 5.1.xx on Linux OS
lately I have found similar steps on following site you can take help of this as well
http://confluence.atlassian.com/display/DISC/Prerequisites+and+installation+on+Fedora+or+RHEL+Linux

first steps to find where current version of MYSQL has been installed
its location of binaries, datafiles and its config file (my.inf/cnf) if any

1. give following command to check MYSQL processes running
 
[root@RAJ ~]# ps ax | grep mysqld

from its output note down location of
mysqld running (/home/raj/mysql/bin/mysqld)
--basedir=/home/raj/mysql (location where mysql binaries are located)
--datadir=/var/lib/mysql (data files of databases created)
--defaults-extra-file =/home/raj/mysql/data/my.cnf (config file if any specifed other than /etc path)

2. now stop the server
 
[root@RAJ ~]# mysqladmin shutdown

3. now fire command in step one again to verify that no process is running.

4. now check which packages are installed of MYSQL
 
[root@RAJ ~]# rpm -qa | grep mysql
mysql-4.1.22-2.el4
mysqlclient10-3.23.58-4.RHEL4.1

5. try to uninstall with rpm command for the above displayaed packags
 
[root@RAJ ~]# rpm -e mysql-4.1.22-2.el4
error: Failed dependencies:
libmysqlclient.so.14 is needed by (installed) perl-DBD-MySQL-2.9004-3.1.centos4.i386
libmysqlclient.so.14 is needed by (installed) cyrus-sasl-sql-2.1.19-14.i386
libmysqlclient.so.14(libmysqlclient_14) is needed by (installed) cyrus-sasl-sql-2.1.19-14.i386


perl-DBD-MySQL = A bundle to install Perl drivers for MySQL 4 since for mysql5 we will not need it
cyrus-sasl-sql = SASL is the Simple Authentication and Security Layer, mysql5 will have in its own package installation

by looking at error we know that there is dependancy by other files so try to remove those files first
if you require pearl for mysql5 there are newer version of perl driver you need to install.
NOTE: I could have removed it with --nodeps parameter which i come to know later so you may try with this parameter as follows
rpm --nodeps --allmatches -e mysql-4.1.22-2.el4
rpm --nodeps --allmatches -e mysqlclient10-3.23.58-4.RHEL4.1


6. remove dependencies (pls check note above)
 
[root@RAJ ~]# rpm -e perl-DBD-MySQL-2.9004-3.1.centos4.i386
[root@RAJ ~]# rpm -e cyrus-sasl-sql-2.1.19-14.i386

7. now try again step 5 again (pls check note above)
 
[root@RAJ ~]# rpm -e mysql-4.1.22-2.el4
warning: /etc/my.cnf saved as /etc/my.cnf.rpmsave

8. remove mysqlclient (pls check note above)
 
[root@RAJ ~]# rpm -e dovecot-0.99.14-1.1.el3.rf.i386
[root@RAJ ~]#
[root@RAJ ~]# rpm -e mysqlclient10-3.23.58-4.RHEL4.1

9. after this is done pls check data and binary folder of mysql is removed or not if not removed then rename it if you require them or can safely remove it

10. I have downloaded following rpm from mysql site
 
[root@RAJ installers]# ls -l
total 25864
-rw-r--r-- 1 root root 7256227 Aug 18 06:02 MySQL-client-community-5.1.37-0.rhel4.i386.rpm
-rw-r--r-- 1 root root 19179293 Aug 18 01:50 MySQL-server-community-5.1.37-0.rhel4.i386.rpm
[root@RAJ installers]# chmod 755 *

11. Install new version of mysql 5 server and client
 
[root@RAJ installers]# rpm -i MySQL-server-community-5_1.1.37-0.rhel4.i386.rpm
warning: MySQL-server-community-5_1.1.37-0.rhel4.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5
090818 7:52:19 [Warning] Forcing shutdown of 2 plugins
090818 7:52:20 [Warning] Forcing shutdown of 2 plugins

It shows some more message on screen and automatically starts mysql server.
 
[root@RAJ installers]# rpm -i MySQL-client-community-5.1.37-0.rhel4.i386.rpm
warning: MySQL-client-community-5.1.37-0.rhel4.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5

this completes the installation of mysql server and clients
there are some post installation steps to be folllowed to configure mysql if needed
like setting up my.cnf file to /etc location or creating auto start script of mysql on start of server. etc.

12. check mysql version now
 
[root@RAJ Whitehedge]# mysql -V
mysql Ver 14.14 Distrib 5.1.37, for pc-linux-gnu (i686) using readline 5.1

Friday, June 26, 2009

Getting started with Ajax: Basics.

Background:

I had used XmlHttp in my javascripts for quite long and extensively. espically dealing with grid control... but lately I come to know that it was called as an AJAX... which has much more than that... to make your UI richer. A common way of calling ajax function is to create an object of XMLHttpRequest. this you can either use it directly or can use wrapper libraries available in the market like extjs, JQuery, Prototype etc. to begin with I will present the internal coding of actual ajax code how it works. please see the following code...

Basic Code: First to check which browser we are using and accordingly activate XmlHttp object as



function GetXmlHttpObject(){
var xmlHttp=null;
try{
// Firefox, Opera 8.0+, Safari
xmlHttp=new XMLHttpRequest();
} catch (e){
// Internet Explorer
try{
xmlHttp=new ActiveXObject("Msxml2.XMLHTTP");
}catch (e){
xmlHttp=new ActiveXObject("Microsoft.XMLHTTP");
}
}
return xmlHttp;
}

It just that I have saparated code. creating of XMLHttp object in different function as above and calling it in other function as below


function getAjaxResponse(sURL){
xmlHttp = GetXmlHttpObject();
xmlHttp.onreadystatechange=stateChanged(){
if (xmlHttp.readyState==4){
document.getElementById(divID).innerHTML=xmlHttp.responseText;
};
};
xmlHttp.open("GET",sURL ,true);
xmlHttp.send(null);
}

Lets observe the code above we have specifed call back function to receive response from server.



xmlHttp.onreadystatechange=stateChanged();

... here you can either mention function name or you can inline the function itself. this function block will get executed when you receive response from server. Here actually you can do more based on the status code that you receive from server, for e.g. you can show process status while you receive response or please wait messgage/image to user till response received etc. please see the table below for the status code that you get on this function.
Response that you get from the server can be of Html code or xml or simply a string, and it depends how you want to show it to user. in above code I have shown that it returns html response which I directly paste into div block.
fire http GET/POST request to server


xmlHttp.open("GET",url,true);
xmlHttp.send(null);

Please note the following things when calling open,

  • Always set Async to true.
  • always set call back function before calling send()
  • To send form data/POST use endcodeURI() for all data values and set mime type of request to "application/x-www-form-urlencoded"

basic advantage of using javascript to post the request to server is that we don't have to wait till server response arrives. instead we can perform other task. I am listing below the XMLHttpRequest objects method available here.


Method/ PropertyDescription.
abort()stops listining for current response.
getAllResponseHeader()Get array of response header as key value pair.
getRespoonseHeader(name)Get response header by name passed as key.
open(method, url[,asyncFlag[, "username"[, "password"]]])

This prepares the xmlHTTPRequest object to make call to the server.
Method: HTTP; can be either GET, PUT or POST.
url: is url string to open can be either obsolute or relate may include querystring

send(content)

perform http request.
content: string to be used as request body, i.e. to form POST body, you can format your POST parameters just like querystring.

setRequestHeader(header, value) Add header to Http requset, normally a key value pair and mostly used to set content type of request.
onreadystatechangeused to set as callback fucntion that handles request state change.(similar to onclick, onload)
readyState

returns the status of the request which can be on of following at a time
0:= uninitialised
1:= loading
2:= loaded
3:= interactive
4:= complete

responseTextreturns server response as String
ResponseXMLreturns server response as XML Document
statusreturns HTTP status code like 200 for OK etc
statusTextreturn String representation of HTTP status code like OK for 200, not found for 404 etc.

In above example I have already shown how I am getting HTML response from server which I am setting back to perticlular html object. see below the example of parsing xml response received from server and setting to html object.
here I assume that I am getting some key-value pair from server as xml response which i need to set to combobox in html so I am passing combobox object to method

xmlHttp.onreadystatechange=usrfunc;
....
....
function processListXML(xmlId, objlist ){
// first remove all existing option values from list box
for(i=objlist.options.length-1;i>=0;i—){
objlist.remove(i);
}
var t = document.getElementById(xmlId);
var tableData = t.getElementsByTagName("RList");
if(tableData != null){
var t = tableData.item(0);
var columns = t.getElementsByTagName("ROption");
if(columns != null){
for (j = 0; j < columns.length; j++){
key= columns[j].getAttribute("id");
value= columns[j].firstChild.data;
objlist.add(new Option( value ,key ));
}
}
}
}


Now assume that I get following xml
<RList>
<ROption id="01">INDIA </ROption>
<ROption id="02">JAPAN </ROption>
<ROption id="03">AMERICA </ROption>
</RList>

Friday, May 22, 2009

Web Service Quick Start

 

What are Web Services?
Web services are application components
Web services communicate using open protocols
Web services are self-contained and self-describing
Web services can be discovered using UDDI
Web services can be used by other applications
XML is the basis for Web services

How Does it Work?
The basic Web services platform is XML + HTTP.

XML provides a language which can be used between different platforms and programming languages and still express complex messages and functions.

The HTTP protocol is the most used Internet protocol.

below are Web services platform elements:

SOAP (Simple Object Access Protocol)
a framework for exchanging XML-based information in a network
It defines a uniform way of passing XML-encoded data.
In also defines a way to perform remote procedure calls (RPCs) using HTTP as the underlying communication protocol.

UDDI (Universal Description, Discovery and Integration)
provides a registry mechanism for clients and servers to find each other
uses SOAP for communication

WSDL (Web Services Description Language)
an XML-based language for describing network services
WSDL descriptions of capabilities and locations of services
like an interface description language for Web services
communication using SOAP or direct HTTP

Web services uses set of tools that can be used in a number of ways. The three most common styles of use are RPC, SOA and REST.
Remote procedure calls
RPC Web services present a distributed function (or method) call interface that is familiar to many developers.
Typically, the basic unit of RPC Web services is the WSDL operation
The first Web services tools were focused on RPC, and as a result this style is widely deployed and supported. However, it is sometimes criticized for not being loosely coupled, because it was often implemented by mapping services directly to language-specific functions or method calls. Many vendors felt this approach to be a dead end, and pushed for RPC to be disallowed in the WS-I Basic Profile.

Service-oriented architecture
Web services can also be used to implement an architecture according to Service-oriented architecture (SOA) concepts, where the basic unit of communication is a message, rather than an operation. This is often referred to as "message-oriented" services.

SOA Web services are supported by most major software vendors and industry analysts. Unlike RPC Web services, loose coupling is more likely, because the focus is on the "contract" that WSDL provides, rather than the underlying implementation details.

Representational state transfer
Representational State Transfer (REST) attempts to describe architectures which use HTTP or similar protocols by constraining the interface to a set of well-known, standard operations (like GET, POST, PUT, DELETE for HTTP). Here, the focus is on interacting with stateful resources, rather than messages or operations. An architecture based on REST (one that is 'RESTful') can use WSDL to describe SOAP messaging over HTTP, which defines the operations, can be implemented as an abstraction purely on top of SOAP (e.g., WS-Transfer), or can be created without using SOAP at all.

 

Further to understand how RPC model can be defined using the following steps:

1. A client application builds an XML document containing the URI of the server that will service the request, the name of the remote method to execute, and the parameters associated with that method.
2. The targeted server receives and unwinds the XML document. It then executes the named method.
3. After the named method has returned its results, the results are packed into a response XML document, which is sent back to the calling client.
4. The client application receives the response and unwinds the results, which contains the response of the invocated method.

Web Services tools with Java
The most essential Java tools for Web service development:
xml.apache.org/axis - Apache AXIS (Apache Extensible Interaction System)
a Java-based implementation of SOAP+WSDL
largely allows the programmer to forget these technologies  typically used together with Tomcat

www.alphaworks.ibm.com/tech/ettk - alphaWorks's EETK (Emerging Technologies Toolkit)
support for SOAP, WSDL, UDDI and much more...

java.sun.com/webservices/downloads/webservicespack.html - Sun's Java WSDP (Web Services Developer Pack)
support for SOAP, WSDL, UDDI, ...
JAX-RPC maps SOAP/WSDL to RMI (Java Remote Method Invocations)

imp links
Tutorial on what is web service
http://www.w3schools.com/webservices/default.asp

Typical soap RPC based web service example
http://www.onjava.com/pub/a/onjava/2002/02/27/tomcat.html

List of web service framework on which web service can be built
http://en.wikipedia.org/wiki/List_of_Web_service_Frameworks

Wikipedia definition and detail note on web service
http://en.wikipedia.org/wiki/Web_service

Top Ten FAQs for Web Services
http://webservices.xml.com/pub/a/ws/2002/02/12/webservicefaqs.html

Some Notes on Coding

some of the points you should note while doing development

1. on java

  • Use primitive data types instead of objects as instance variables.
  • Use the final modifier on instance-variable definitions to create immutable internally accessible objects
  • Eliminate unnecessary casts (use jdk1.5 feature)
  • String.equals() is expensive if you are only testing for an empty string. It is quicker to test if the length of the string is 0.
  • When using Vector, ensure that elementAt() is not used inside a loop.
  • Modify java.lang.String to cache the hashCode if you are using many string keys in hash tables [note Sun added this optimization to the String class in SDK 1.3]
  • Reorder CLASSPATH so that the most used libraries occur first
  • Compile java files with the optimizer on.

 

2. On JDBC

  • Turn off autocommit, but don't leave transactions open for too long.
  • Cache any required metadata and use metadata methods as rarely as possible as they are quite slow. Use Connection.setReadOnly(true) to optimize read-only
  • Choose the optimal cursor: forward-only for sequential reads; insensitive for two-way scrolling. Avoid insenstive cursors for queries that only return one row.
  • Use a parametrized remote procedure call (RPC) rather than passing parameters as part of the RPC call, e.g. use Connection.prepareCall("Call getCustName (?)").setLong (1,12345) rather than Connection.prepareCall("Call getCustName (12345)")
  • CachedRowSet provides cached result sets that do not require continuous connection to the database, allowing connections to be reused more efficiently.
  • Use PreparedStatements to batch statements for optimal performance.

 

3. On JSP/Servlet

  • Avoid using the SingleThreadModel interface for servlets: write thread-safe code instead.
  • ServletRequest.getRemoteHost() is very inefficient, and can take seconds to complete the reverse DNS lookup it performs.
  • OutputStream can be faster than PrintWriter. JSPs are only generally slower than servlets when returning binary data, since JSPs always use a PrintWriter, whereas servlets can take advantage of a faster OutputStream.
  • Do not store large object graphs in javax.servlet.http.HttpSession. Servlets may need to serialize and deserialize HttpSession objects for persistent sessions, and making them large produces a large serialization overhead.
  • Call HttpSession.invalidate() to clean up a session when you no longer need to use it.
  • For Web pages that don't require session tracking, save resources by turning off automatic session creation using: <%@ page session="false"%>
  • Timeout sessions more quickly by setting the timeout or using session.setMaxInactiveInterval().
  • Use the include directive <%@ include file="copyleft.html" %>where possible, as this is a compile-time directive (include action is a runtime directive).
  • Use the jspInit() method to cache static data, and release them in the jspDestroy() method.

web performance: have you tried this?

DEFER Your Scripts

DEFER is a relatively obscure attribute of the script element, but the performance-minded page author can use it to indicate to Internet Explorer 4.0 or later that the script tag contains no immediately executing code that impacts the document before the load event fires. Combined with the SRC attribute, the DEFER attribute can be used on a script tag to indicate to Internet Explorer that the associated code should be downloaded in the background while the rest of the components of the page are downloaded and parsed.

<SCRIPT DEFER>
function test1()
{
   // function which is not required at time page load/start  immediately

}
</SCRIPT>

 

Author Hyperlinks Consistently

The Internet Explorer cache is case-sensitive. That means that you should author your hyperlinks with case-sensitivity in mind. Take the following example.
<A  HREF="/workshop/author/dhtml/reference/dhtmlrefs.asp">DHTML References</A>
<A HREF="/Workshop/Author/DHTML/Reference/DHTMLRefs.asp">DHTML References
Both hyperlinks refer to the same page. Or do they?
On a UNIX system these hyperlinks might refer to two distinct pages; thus, Internet Explorer treats them distinctly by making separate requests to the server, allowing the server to decide how to resolve the request.

By authoring your hyperlinks consistently, you'll be saving space in the user's cache, and you'll be reducing the number of times Internet Explorer has to request the same resource from the server.

Close Your Tags

Unlike XML, HTML has the notion of implicitly closed tags. This includes frame, img, li, and p. If you don't close these tags, Internet Explorer renders your pages just fine. If you do close your tags, Internet Explorer will render your pages even faster.

It is tempting to author in the following way.

<P>The following is a list of ingredients.
<UL>
<LI>flour
<LI>sugar
<LI>butter
</UL>But the following will be parsed more quickly because it is well-formed and Internet Explorer doesn't need to look ahead to decide where the paragraph or list items end.

<P>The following is a list of ingredients.</P>
<UL>
<LI>flour</LI>
<LI>sugar</LI>
<LI>butter</LI>
</UL>

 

Leverage the HTTP Expires Header

The expires header is part of the HTTP 1.0 specification. When an HTTP server sends a resource such as an HTML page or an image down to a browser, the server has the option of sending this header and an associated time stamp as part of the transaction. Browsers typically store the resource along with the expiry information in a local cache. On subsequent user requests for the same resource, the browser can first compare the current time and the expires time stamp. If the time stamp indicates a time in the future, the browser may simply load the resource from the cache rather than retrieving the resource from the server.

Even when a resource would advertise an expiration date still to come, browsers?including Internet Explorer 4.0?would still perform a conditional GET to determine that the version of the object in the cache was the same as the version on the server. Upon careful analysis, the designers of Internet Explorer determined that this extra round trip was neither optimal nor necessary. For that reason, the behavior of Internet Explorer 5 has been modified in the following way: if the expiry of a cached resource is later than the time of the request, Internet Explorer will load the resource directly from the cache without contacting the server. Sites using the expires header on commonly used but infrequently updated resources will experience lower traffic volumes, and customers using Internet Explorer 5 will see pages render more quickly.

 

how to make sure updated .js files gets refreshed on client machine without clearing temp folder?

normally js files used in web application, are cached in clients machine and suppose if we next release
with some updation to same file on server, client browser will not come to know this change.
So we have to clean its temporary folder or clear browser cache by going into internet option>> tools >> clear brwoser history button explicitly.

but how to avoid this? how to make sure that every time we release or update .js files on server, the newer copy or updated copy gets loaded in clients browser?

here is the way
<script type="text/javascript" src="./javascript/MyScript.js"></script>
this is the normal way we use script tag which
we are going to replace as

<script type="text/javascript" src="./javascript/MyScript.js?v=14-05-2009at16:11:19"></script>
include date and timestamp with filename so that next time it will have different date and time
and it will not find same in cache so that it will load new one....

Monday, May 18, 2009

Replication... small note

What is Replication



In Simple words, Replication is the simply process of copying data from one location to another.In MySQL database server, Master is main server from which data to be copied and Slave is the server on which data copied.Replication is asynchronous - means your replication slaves do not need to be connected permanently to receive updates from the master,depending on the configuration, you can replicate all databases, selected databases, or even selected tables within a database.
Some of the most common reasons for using replication are described as follows:
Performance :
This is where user can access shared data with different server, which means distributing load from main server, and also improves performance of the main system
Separating Data Entry and Reporting: Working in an environment in which the same database is used for data entry and reporting,you probably know that things aren't always rosy. sometimes you may see that tables are locked for transaction updates, or some of costly reporting queries slow down database performance byusing its resources, Replication provides great help for these issues in which translations are fired on main database (master)while reports are being separated to run on replicated database (slave), which internally improves performance at both endsfor transactions as well as reports.(this solution is best suited if you are not using reports which requires real time data of few sec/min ago or so)
Providing High Availability: Occasionally, you might consider using replication for high availability; that is, to replicate transactions from the main server to a standby server. If the main server fails, you can then point your data sources to the standby server.Replication does not provide any sort of automatic fail-over by itself.Be aware that using replication for high availability takes careful planning and testing.
Mass deployment: Replication can be used to distribute data over multiple regional locations. Then, applications can access various regional servers instead of accessing one central server. This configuration can reduce network load dramatically.
Reduce Data loss to backup: Replication doesn't mean backup of the data. but certainly it can be an alternative to backup data to minimize data lossReplication is the only technology that can satisfy the needs of the most demanding systems, as only replication can provide instant access to data and zero data loss (minimal Recovery Point Objective or RPO and minimal Recovery Time Objective or RTO). Backup is still the cornerstone of a solid disaster recovery strategy, but since backups are usually run only once a day there is a high risk of losing large amounts of data and also having systems being offlinefor long periods of time (the Recovery Point Objective and Recovery Time Objective for backup is usually days, whereas with replication it is usually minutes or seconds).

As we know, backup process always requires data locks on database server while in process (so that no data loss happens) and so it is difficult to take database backup from online transactional database during day (which cause data loss in failure)but with replication we can reduce this problem in a way that

1 main database is free, and need not to be locked during backup process (which gives high avaibility 24x7)

2 Replication gives exact snapshot of main database and so can be used by backup process to take backup instead of connecting to main server

3. Replication gives choice to use as alternate system in fail over or recover data from (slave/backup ) during system down.



imp links for further reading
http://msdn.microsoft.com/en-us/library/aa164940(office.10).aspx

Why Use Replication?

http://www.ameinfo.com/48329.html
for details of implementation and usage in MySQL

http://dev.mysql.com/doc/refman/6.0/en/replication.html

replication on Linux tutorial

http://librenix.com/?inode=8118

Tuesday, April 14, 2009