Monday, July 8, 2013

solved error - zabbix cinfigured with PostgreSQL

decided & installing zabbix tool for monitoring rdbms / no-sql database.

I got few error while installing zabbix with  PostgreSQL database. here i'm listing few with solution.

I created database manually in psq.

CREATE DATABASE zabbix;

ERROR 1)
....
....

pg_free_result() expects parameter 1 to be resource, boolean given [include/db.inc.php:566]

....
 
SOLUTION
I checked database "zabbix" which i was created by manually, no relation were found.
run the following command to create relation schema 


postgres@boss:[~]$ psql -hlocalhost zabbix
psql (9.0.6, server 9.0.0)
Type "help" for help.

zabbix=# \i /usr/share/zabbix-server-pgsql/schema.sql
zabbix=# \i /usr/share/zabbix-server-pgsql/images.sql
zabbix=# \i /usr/share/zabbix-server-pgsql/data.sql

ERROR 2)
....
....

zabbix server is not running the information displayed may not be current

....
 
SOLUTION

step 1) go through the log 
root@boss[~]#vi /var/log/zabbix/zabbix_server.log psql

19447:20130704:162637.902 Database is down. Reconnecting in 10 seconds.
 19447:20130704:162647.903 [Z3001] connection to database 'zabbix' failed: [0] FATAL:  role "zabbix" does not exist 

step 2) log clearly tells that no role in name of zabbix  exist in database

postgres@boss:[~]$ psql -hlocalhost zabbix
psql (9.0.6, server 9.0.0)
Type "help" for help.

zabbix=#CREATE ROLE zabbix WITH LOGIN; 

ERROR 3)
....
....
 19447:20130704:163227.949 Database is down. Reconnecting in 10 seconds.
 19447:20130704:163237.950 [Z3001] connection to database 'zabbix' failed: [0] FATAL:  role "zabbix" is not permitted to log in

may get above error if you don't have log-in rights,
SOLUTION

zabbix=#ALTER ROLE zabbix LOGIN; 

ERROR 4)
You are not able to choose some of the languages, because locales for them are not installed on the web server.

SOLUTION
  its waring message only, to  diaplsy installed locales

root@boss:[~]$locale -a 
to select & install missing locales by
root@boss:[~]$dpkg-reconfigure locales

ERROR 5)
....
....
25046:20130705:142910.383 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation config
 [select alert_history,event_history,refresh_unsupported,discovery_groupid,snmptrap_logging,severity_name_0,severity_name_1,severity_name_2,severity_name_3,severity_name_4,severity_name_5 from config where 1=1 and configid between 0 and 99999999999999]
 25046:20130705:142910.383 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation items
 [select i.itemid,i.hostid,h.proxy_hostid,i.type,i.data_type,i.value_type,i.key_,i.snmp_community,i.snmp_oid,i.port,i.snmpv3_securityname,i.snmpv3_securitylevel,i.snmpv3_authpassphrase,i.snmpv3_privpassphrase,i.ipmi_sensor,i.delay,i.delay_flex,i.trapper_hosts,i.logtimefmt,i.params,i.status,i.authtype,i.username,i.password,i.publickey,i.privatekey,i.flags,i.interfaceid,i.lastclock from items i,hosts h where i.hostid=h.hostid and h.status in (0) and i.status in (0,3) and i.itemid between 0 and 99999999999999]
 25046:20130705:142910.384 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation hosts
 [select distinct t.triggerid,t.description,t.expression,t.error,t.priority,t.type,t.value,t.value_flags from hosts h,items i,functions f,triggers t where h.hostid=i.hostid and i.itemid=f.itemid and f.triggerid=t.triggerid and h.status in (0) and i.status in (0,3) and t.status in (0) and t.flags not in (2) and h.hostid between 0 and 99999999999999]
 25046:20130705:142910.385 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation trigger_depends
 [select d.triggerid_down,d.triggerid_up from trigger_depends d where 1=1 and d.triggerid_down between 0 and 99999999999999 order by d.triggerid_down]
 25046:20130705:142910.388 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation hosts

 25063:20130705:143048.010 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest
 [select min(t.nextcheck) from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 0 and 99999999999999]
 25069:20130705:143049.903 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation escalations
 [select escalationid,actionid,triggerid,eventid,r_eventid,esc_step,status,nextcheck from escalations where 1=1 and escalationid between 0 and 99999999999999 order by actionid,triggerid,escalationid]
 25069:20130705:143052.904 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation escalations
 [select escalationid,actionid,triggerid,eventid,r_eventid,esc_step,status,nextcheck from escalations where 1=1 and escalationid between 0 and 99999999999999 order by actionid,triggerid,escalationid]
 25063:20130705:143053.011 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest
 [select t.httptestid,t.name,t.macros,t.agent,t.authentication,t.http_user,t.http_password from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and t.nextcheck<=1373014853 and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 0 and 99999999999999]
 25063:20130705:143053.012 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest
 [select min(t.nextcheck) from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 0 and 99999999999999]
 25069:20130705:143055.904 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation escalations
 [select escalationid,actionid,triggerid,eventid,r_eventid,esc_step,status,nextcheck from escalations where 1=1 and escalationid between 0 and 99999999999999 order by actionid,triggerid,escalationid]
 25063:20130705:143058.013 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest

[0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation escalations
 [select escalationid,actionid,triggerid,eventid,r_eventid,esc_step,status,nextcheck from escalations where 1=1 and escalationid between 0 and 99999999999999 order by actionid,triggerid,escalationid]
 25064:20130705:143105.801 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation drules
 [select distinct r.druleid,r.iprange,r.name,c.dcheckid from drules r left join dchecks c on c.druleid=r.druleid and uniq=1 where r.proxy_hostid is null and r.status=0 and (r.nextcheck<=1373014865 or r.nextcheck>1373014865+r.delay) and mod(r.druleid,1)=0 and r.druleid between 0 and 99999999999999]
 25064:20130705:143105.801 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation drules
 [select count(*),min(nextcheck) from drules where proxy_hostid is null and status=0 and mod(druleid,1)=0 and druleid between 0 and 99999999999999]
 25060:20130705:143105.927 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation alerts
 [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.message,a.status,mt.mediatypeid,mt.type,mt.description,mt.smtp_server,mt.smtp_helo,mt.smtp_email,mt.exec_path,mt.gsm_modem,mt.username,mt.passwd,a.retries from alerts a,media_type mt where a.mediatypeid=mt.mediatypeid and a.status=0 and a.alerttype=0 and mt.mediatypeid between 0 and 99999999999999 order by a.alertid]
 25069:20130705:143107.907 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation escalations
 [select escalationid,actionid,triggerid,eventid,r_eventid,esc_step,status,nextcheck from escalations where 1=1 and escalationid between 0 and 99999999999999 order by actionid,triggerid,escalationid]
 25063:20130705:143108.016 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest
 [select t.httptestid,t.name,t.macros,t.agent,t.authentication,t.http_user,t.http_password from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and t.nextcheck<=1373014868 and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 0 and 99999999999999]
 25063:20130705:143108.017 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  permission denied for relation httptest

SOLUTION
to check the owner of the relation under database zabbix, it must be zabbix. in my case it is public then I alter into zabbix

step 1) check the owner of the table in PostgreSQL by


postgres@boss:[~]$ psql -hlocalhost zabbix
psql (9.0.6, server 9.0.0)
Type "help" for help.

zabbix=#\dt             
 List of relations
 Schema |         Name          | Type  | Owner  
--------+-----------------------+-------+--------
 public | acknowledges          | table | public
 public | actions               | table | public
 public | alerts                | table | public
 public | applications          | table | public
 public | auditlog              | table | public
 public | auditlog_details      | table | public
 public | autoreg_host          | table | public 
 ....
....
here Owner of the relation is public, alter into zabbix by

its tedious to alter one by one through single alter statement by

postgres@boss:[~]$ psql -hlocalhost zabbix
psql (9.0.6, server 9.0.0)
Type "help" for help. 
 alter table  owner to zabbix;
instead of,  alter owner of the all relation in zabbix database in PostgreSQL by

postgres@boss:[~]$ for tbl in `psql -qAt -h localhost -c "select tablename from pg_tables where schemaname = 'public';" zabbix` ;
 do  psql -hlocalhost -c "alter table $tbl owner to zabbix" zabbix ; done

ERROR 6)
item [Zabbix server:zabbix[process,ipmi poller,avg,busy]] became not supported: No "ipmi poller" processes started
 25066:20130705:145544.791 item [Zabbix server:zabbix[process,java poller,avg,busy]]  became not supported: No "java poller" processes started
 25066:20130705:145544.791 item [Zabbix server:zabbix[process,node watcher,avg,busy]] became not supported: No "node watcher" processes started
 25066:20130705:145549.798 item [Zabbix server:zabbix[process,snmp trapper,avg,busy]] became not supported: No "snmp trapper" processes started abbix-server-pgsql/data.sql

SOLUTION
open zabbix_server.conf file and add below 2 line

root@boss:[~]$ vi /etc/zabbix/zabbix_server.conf
StartSNMPTrapper=1
SNMPTrapperFile=[TRAP FILE]


ERROR 7)
PHP Fatal error:  Call to undefined function mysql_connect() in /etc/zabbix/mysql.php on line 62ERROR

SOLUTION

 open and add  extensions=mysql.so
root@boss:[~]$ vi /etc/php5/apache2/php.ini 
 extensions=mongo.so
and 
root@boss:[~]$ vi /etc/php5/cli/php.ini 
extensions=mongo.so


































Tuesday, July 2, 2013

Resolved Issue / Error in sqoop 1.4

I've posted some issue in sqoop2  here.

I had a good time when working with sqoop 1.4 connecting to PostgreSQL  database. I have faced few issues & managed to overcome those issues.

ERROR 1)
....
....
sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: 
Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver 

SOLUTION
download & put MySQL connector to the $SQOOP_HOME/lib dir.


$SQOOP_HOME/lib               

ERROR 2)
....
....
sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
.....
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
.....
Caused by: java.net.ConnectException: Connection refused
SOLUTION
check the status of the MySQL up & running.ensure user has rights to access the database.for more find here.

step 1)  verify that you can connect to the database from the node where you are running Sqoop:


root@boss[~]#mysql --host=localhost --database=test --user=root --password=root
step 2) Grant permissions to the user to access the database

root@boss[~]#mysql --host=localhost --user=root --password=root
mysql>grant all privileges on test.* to 'sqoopuser'@'%' identified by 'sqooppassword'

ERROR 3)
....
....
orm.ClassWriter: No parser available for Java type org.apache.hadoop.io.BytesWritable
SOLUTION
  in postgresql/oracle , column with data type bytea/binary or varbinary  contains null value means through above error.
update the "bytea/binary or varbinary" column with not-null value.              

ERROR 4)
....
....
INFO ipc.Client: Retrying connect to server: /127.0.0.1:8021. Already tried 0 time(s).   
 ERROR tool.ImportTool: Encountered IOException running import job: java.net.ConnectException: 
 Call to /127.0.0.1:8021 failed on connection exception: java.net.ConnectException: Connection refused
 ERROR 5)
....
....
ERROR tool.ImportTool: Encountered IOException running import job: 
java.io.IOException: Call to /127.0.0.1:8021 failed on local exception: 
java.io.IOException: Connection reset by peer
 SOLUTION (4,5)
check the status of hadoop process
root@boss[~]#jps  
20668 JobTracker
20434 DataNode
20299 NameNode
20569 SecondaryNameNode
20799 TaskTracker20844 Jps
ensure all the 5 process are running successfully. to avoid the error 5) JobTracker must be up & running.


ERROR 6)
....
....
localhost: ssh: Could not resolve hostname localhost: Name or service not known

ERROR 7)
....
....
ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
java.net.ConnectException:
 Call to localhost/127.0.0.1:9000 failed on connection exception: 
java.net.ConnectException: Connection refused
SOLUTION (6,7)
I've faced said error while running hbase, to resolve the error, open & edit  /etc/hosts

ip/domain
127.0.0.1    localhost.localdomain
127.0.0.1    boss
127.0.0.1    hbase-master:q
 to resolve the error 7)
127.0.0.1    hbase-master:q
must be added in file /etc/hosts

I've connected PostgreSQL database using sqoop, tried out store the data across HDFS and analysis using hbase. 


Monday, July 1, 2013

ERROR working with Sqoop 1.99.2


Operating System : Debian / Ubuntu / BOSS Gnu Linux
pre-requisite : HBASE_HOME & ZOOKEEPER_HOME must be set in /etc/profile

while running  sqoop, I faced following error's, I resolved one but not all bec. current version of sqoop2 not compatible with apache hadoop & its not stable.

ERROR 1)


Exception has occurred during processing command 
Exception: com.sun.jersey.api.client.UniformInterfaceException Message: 
 GET http://localhost:12000/sqoop/v1/connector/all 
returned a response status of 404 Not Found

SOLUTION
set server properly


set server --host localhost --port 8050 --webapp sqoop
once I did above step, i got stuck by 
ERROR 2)

Exception has occurred during processing command 
Exception: com.sun.jersey.api.client.ClientHandlerException Message: 
java.net.ConnectException: Connection refused

ERROR 3)

Exception has occurred during processing command 
Exception: com.sun.jersey.api.client.UniformInterfaceException Message: 
GET http://10.184.39.34:80/sqoop

still this bug not resolved as sqoop2 is still in development.