Big Data

I'm currently learning what this Big Data specifically using Apache's Hadoop is all about and I can say its a a Big WOW just by imagining how much data it can process. If you're an IT guys like me that has Database administration background using MS-SQL, MySQL, Oracle, etc, you would ask how those all the data (syslog, weblog, sales, etc.) can be imported to a database and make use of it or produce information that can help the business or customers.

Big Data can use or import using Pig (can eat anything) or if you're not much of a java programmer, a Hive can be use to create a MapReduce jobs. There are also tools like Sqoop and Flume that can help importing file or streaming data.

Good thing also about Hadoop is that the License is under Apache, which means no one OWNS it and the public community can use it, for Free.

Link:

http://hadoop.apache.org

Python 101

With SDN on the rise, network engineer should know how to use either Java or Python in able to manage the Controller to talk to the API.

Google has a basic Python class available to begin with:

https://developers.google.com/edu/python/


Happy Scripting!!!


UPDATE: Playing with Python..

With the recent Earthquake in the Philippines last January 11, 2015 at around 3:30AM (Phil. Time) that reach 5.9 Magnitude, I though maybe I can create an image (Data Visualisation) on where is the centre of the earthquake. So here's python to the rescue. Since I been reading for awhile regarding what other modules can do, I ended up using Basemap. The script below has 2 parts. First is to grab dataset from the cvs file. The columns are Latitude, Longitude and Magnitude. The second part is the engine for the script where Baseman is being used to create the image.

For the result, the red dot will got bigger if the magnitude is higher.

----------------------------------------------------------------

# Import Dataset

import csv

filename ='DS.csv'

lats, lons = [], []

mags = []

with open(filename) as f:

    # Create a csv reader object.

    reader = csv.reader(f)

    # Ignore the header row.

    #next(reader)

    # Store the latitudes and longitudes in the appropriate lists.

    for row in reader:

        lats.append(float(row[0]))

        lons.append(float(row[1]))

        mags.append(float(row[2]))

#----------------------------------------------------------------

from mpl_toolkits.basemap import Basemap

import matplotlib.pyplot as plt

import numpy as np

## Philippine Map coordinates

m = Basemap(resolution='f',projection='merc',                                        

            lat_0=13, lon_0=122,

            llcrnrlat=5.0,                                                         

            urcrnrlat=19.0,                                                          

            llcrnrlon=114.,                                                          

            urcrnrlon=130.0,                                                         

            ) 

m.drawmapboundary(fill_color='white')                                                

m.fillcontinents(color='#F5DEB3',lake_color='#85A6D9')                               

m.drawcoastlines(color='black', linewidth=.4)                                        

m.drawcountries(color='#6D5F47', linewidth=.4)                                       

min_marker_size= 2.5

for lon, lat, mag in zip(lons, lats, mags):

    x,y = m(lon, lat)

    msize = mag * min_marker_size 

    m.plot(x,y, 'ro', markersize=msize)

figsize=(50, 18)

plt.show()

DS.csv

09.97,124.17,2.6

11.64,126.10,3.2

11.55,126.31,3.1

11.59,126.23,4.9

04.87,127.19,3.7

06.17,126.02,2.5

14.79,120.00,2.3

05.69,126.26,4.2

14.74,119.91,5.9

Final image:

I'm Back.....

Wow, it's been years since I last blog. Since then, I finished my CCNP for Routing and Switching as well as for Security. I'm also been active CISSP since then.

Now for my biggest challenge of all..CCIE. I can say that this exam is a Beast. I've built my ccie home lab from old Cisco routers and switches and it's been a year since I started studying. The good thing about it is that the more I study, the more I know that there are lots of things to learn, and the the more things I've learn, the more I can design network efficiently and securely.

I'll be posting more configuration setup here for me to refresh my memory and hopefully help others as well.....

Happy Studying.....