Import .csv file into MySQL database

Aug 22 2010

I was implementing PhishTank database to one of my projects.

PhishTank is a service, operated by OpenDNS which updates a current list of phishing sites on the Internet.

When dealing with large number or requests it is advised that you download their hourly updated database dump.

I decided to grab the .csv dump and import it in my database using cron.

The heart of this cron is the following command which imports .csv data into local database

mysql -u$username -p$password -e \
"load  \
    data local
infile  \
    \"./verified_online.csv\" \
into table  \
    $database.$table \
fields terminated by \",\" \
enclosed by '\"' \
lines terminated by \"\n\" \
( \
    phish_id, \
    phish_detail_url, \
    url, \
    submission_time, \
    verified, \
    verification_time, \
    online, \
    target \
);"

Make sure that you already have the table created with following fields “phish_id, phish_detail_url, url, submission_time, verified, verification_time, online, target”

No responses yet

Switch SVN local copy to new server location

Aug 19 2010

When an SVN repository switches the server you need to update your working copy with new repository url.
First, you need to get the old repository url by typing ‘svn info’. You will get a listing saying ‘URL: http://oldserver/path/to/repo/trunk’.

svn info
URL: http://oldserver/path/to/repo/trunk

Copy that url and execute the following from the root folder of your local copy

cd /path/to/your/local/copy/
svn switch --relocate \
http://oldserver/path/to/repo/trunk \
http://newserver/path/to/repo/trunk \
. \

After that update your local copy and you’re ready to go

svn update

No responses yet

Mount Windows Share folder on Ubuntu

Aug 17 2010

In order to mount remote shared folder ‘/svn/foo’ from remote host 192.168.1.2 to ‘foo’ folder on your Desktop do this:

mount -t cifs \
//192.168.1.2/svn/foo \
-o username=MyUsername,password=MyPassword \
/home/haris/Desktop/foo

No responses yet

PHP method chaining

Jul 31 2010

Method chaining is a technique that adds class methods one on top the other. It’s commonly used in jQuery and models in PHP frameworks.

The whole “magic” behind object chaining is made with a single line – “return $this” which is added in each method you wish to chain.

So, in order to get this behavior in PHP:

$human = new Human;
$human->setName('Haris')
      ->setAge(30)
      ->setSex('male')
      ->display();

you have to define class(es) like this:

class Human {

    protected $_name;
    protected $_age;
    protected $_sex;

    public function setName($name) {
        $this->_name = $name;
        return $this;
    }

    public function setAge($age) {
        $this->_age = $age;
        return $this;
    }

    public function setSex($sex) {
        $this->_sex = $sex;
        return $this;
    }

    public function display() {
        printf('Hi, I\'m %s. I\'m %d years old, and I\'m %s!',
            $this->_name,
            $this->_age,
            $this->_sex
        );
    }

}

class Man extends Human {
    protected $_sex = 'male';
}

class Woman extends Human {
    protected $_sex = 'female';
}

Now compare the methods called with method chaining and the old fashioned way

$woman = new Woman;
$woman->setName('Jane')
      ->setAge(27)
      ->display();

$woman = new Woman;
$woman->setName('Jane');
$woman->setAge(27);
$woman->display();

// Hi, I'm Jane. I'm 27 years old, and I'm female!

No responses yet

Dynamic apache vhosts

Jun 28 2010

As a developer you most likely run dozens of projects in your webroot directory.

Setting up a new apache host and meddling through the apache conf files for every single project is just a waste of time. Not mentioning that conf file(s) can go huge over time. So, for the past six months or so I’ve been using a dynamic vhost configuration.

The concept is very simple. Let’s assume that you want to access each project via some dynamic domain like “$project.local”. “$project” would be a dynamic project name, and “.local” would actually be a localhost server.

First thing first, make sure that vhost module is enabled:

$ sudo cp /etc/apache2/mods-available/mod_vhost_alias.load /etc/apache2/mods-enabled/mod_vhost_alias.load

Now edit the default apache conf file:

$ sudo nano /etc/apache2/sites-enabled/000-default

Add following configuration to it:

<VirtualHost *:80>
ServerName local
ServerAlias *.local
VirtualDocumentRoot /var/www/%1/public_html
UseCanonicalName Off
</VirtualHost>

Retart the apache service

$ sudo service apache2 restart

Now the only “hard” part about this dynamic apache vhost configuration is that you have to set a new line ih hosts file for each project, which will point to local ip address. I’ve heard that this can also be dynamically ser over the bind DNS server, but I’ll explore that some other time

$ sudo nano /etc/hosts
127.0.0.1   local
127.0.0.1   foo.local

No responses yet

Rsync using SSH RSA auth key

Sep 23 2009

rsynclogo

How to rsync files between two servers when remote server uses an RSA key for authentication.

Format:
rsync $options $host:$from $to

rsync -avz -e "ssh -i key.rsa" \
user@domain.com:/var/www /var/www/backup

The ‘-avz’ options are used for creating archive, being verbose and compressing the file transfer.

‘-e “$command”‘ specifies the shell command to use; in this case calling the SSH RSA key used for authentication

No responses yet

Resize and position Gnome Terminator

Sep 12 2009

gnome-terminator

I am a huge fan of Gnome Terminator. Terminator is great because it allows multiple terminal instances to be run in a single window.

Since I usually run two or three separate terminal windows, I ofter find myself resizing the main window – so other istances can fit nicely. This can get very boring and time consuming when you need to do something on the fly.

After snooping around the help files, I found this nice little parameter which resizes and positions the Terminator window on my screen.

/usr/bin/terminator --geometry=1024x500+128+200

This command line can be added to Gnome-do or Gnome panel shortcuts, and it works great.

The first two parameters (1024×500) set the width and height, and the other two(128+200) set the padding from left and top of the screen.

No responses yet

Running both MySQL 4 and 5 on a same box

Apr 06 2009

mysql

I had to make a clone of a website on a development server. The site was running on MySQL 4 and dev server on version 5. Ideally, all queries would run fine, but a huge query with lots of joins was failing for some reason.

In order to keep a clone as is, I decided to try and install MySQL 4 along with version 5 on the same server.

On dev server MySQL 5 was installed using apt-get, and I installed MySQL 4 like this:

Download MySQL 4 source from a local mirror

$wget http://dev.mysql.com/get/Downloads/MySQL-4.1/mysql-4.1.22.tar.gz/from/http://mysql.blic.net/

Unpack and install

$tar zxvf mysql-4.1.22.tar.gz
$sudo groupadd mysql
$sudo useradd -g mysql mysql
$cd mysql-4.1.22
$./configure --prefix=/usr/local/mysql
$make
$sudo make install
$sudo cp ./support-files/my-medium.cnf /etc/my.cnf
$cd /usr/local/mysql
$sudo ./bin/mysql_install_db --user=mysql
$sudo chown -R root .
$sudo chown -R mysql var
$sudo chgrp -R mysql .

Start the daemon

$sudo /usr/local/mysql/bin/mysqld_safe --user=mysql --port=3308 --sock=/tmp/mysql4.sock &

Connect to installed MySQL 4 using shell

$/usr/local/mysql/bin/mysql --socket=/tmp/mysql4.sock

Display database version

mysql> select version();
+------------+
| version() |
+------------+
| 4.1.22-log |
+------------+

To connect phpMyAdmin to MySQL 4 edit the phpMyAdmin config file and change server value ‘localhost’ to ‘:/tmp/mysql4.sock

From web interface just type ‘mysql’ for username, and leave password empty. (Do set a password for MySQL user later, this was just an example)

No responses yet

OpenDNS

Mar 28 2009

opendns

I needed some good content filtering service (anti-pr0n, anti-phishing, anti-….). So, I googled for “open” (which I do a lot lately when looking for software or services :)), and “dns”. Was taken to OpenDNS and I fell in love with it immediately.

Their site looks really good and displays the most crucial information on a single page, which i like a lot.

I signed up for the account, and this is how I set up my computer to use OpenDNS.

Add OpenDNS nameservers to resolv.conf file

$sudo nano /etc/resolv.conf
# OpenDNS
nameserver 208.67.222.222
nameserver 208.67.220.220

My ISP DNS names are always forced to resolv.conf when I connect to Internet, so I had to protect this file from being overwritten by network manager (i – immutable)

$sudo chattr +i /etc/resolv.conf

Install ddclient (updates IP addresses at dynamic DNS services) on Ubuntu

$sudo apt-get install ddclient

Configure ddclient

$sudo dpkg-reconfigure ddclient
Dynamic DNS service provider: other
Dynamic DNS server: updates.opendns.com
Dynamic DNS update protocol: dyndns2
DynDNS fully qualified domain names: $label //this is actually a network "label" set on the OpenDNS "networks" page
Username for dynamic DNS service: $username
Password for dynamic DNS service: $password
Interface used for dynamic DNS service: ppp0 // I use network manager to connect to DLS provider, which creates a ppp0 interface
Run ddclient on PPP connect: yes
Run ddclient as daemon: yes
ddclient update interval: 300

Check that ddclient daemon is up and running

ps aux | grep ddclient

Make sure to add “ssl” option to ddclient config file, otherwise OpenDNS will not be updated

sudo nano /etc/ddclient.conf
# /etc/ddclient.conf
ssl=yes

Restart ddclient

$sudo /etc/init.d/ddclient restart

This setup will check if my IP address has been changed every 5 minutes. If changed, my account on OpenDNS will be automatically updated.

I switched both home and office computers to OpenDNS, and so far all I have are words of praise.

No responses yet

Arctic Silver 5

Mar 22 2009

Arctic Silver 5

A fan on a CPU heatsink was causing me trouble lately. The fan made a weird noise while its operating temperature was low, as if someone was pressing a finger against it.

I pulled out the faulty part to get the part number and made an order of a new heatsink. While inspecting the faulty part, I noticed that a thermal paste on the bottom was in a really bad shape.

So, from another computer I googled up for a good thermal paste. Read a couple of the comparison tests, and it did not took long to reach the Arctic Silver 5 web page.

I ordered a small package of 3.5grams, and after applying it once, enough of it is left for at least five more coatings.

After applying the paste to a new heatsink (identical to a broken one), the temperature in processor cores immediately fell down by 10℃. I was more then impressed.

Detailed instructions for applying the paste per processor type can be found on their website in pdf, along with the reviews and comparison sheets on the review pages.

From now on, I am regular.

No responses yet

Older »