Big Data

I'm currently learning what this Big Data specifically using Apache's Hadoop is all about and I can say its a a Big WOW just by imagining how much data it can process. If you're an IT guys like me that has Database administration background using MS-SQL, MySQL, Oracle, etc, you would ask how those all the data (syslog, weblog, sales, etc.) can be imported to a database and make use of it or produce information that can help the business or customers.

Big Data can use or import using Pig (can eat anything) or if you're not much of a java programmer, a Hive can be use to create a MapReduce jobs. There are also tools like Sqoop and Flume that can help importing file or streaming data.

Good thing also about Hadoop is that the License is under Apache, which means no one OWNS it and the public community can use it, for Free.

Link:

http://hadoop.apache.org

Python 101

With SDN on the rise, network engineer should know how to use either Java or Python in able to manage the Controller to talk to the API.

Google has a basic Python class available to begin with:

https://developers.google.com/edu/python/


Happy Scripting!!!


UPDATE: Playing with Python..

With the recent Earthquake in the Philippines last January 11, 2015 at around 3:30AM (Phil. Time) that reach 5.9 Magnitude, I though maybe I can create an image (Data Visualisation) on where is the centre of the earthquake. So here's python to the rescue. Since I been reading for awhile regarding what other modules can do, I ended up using Basemap. The script below has 2 parts. First is to grab dataset from the cvs file. The columns are Latitude, Longitude and Magnitude. The second part is the engine for the script where Baseman is being used to create the image.

For the result, the red dot will got bigger if the magnitude is higher.

----------------------------------------------------------------

# Import Dataset

import csv

filename ='DS.csv'

lats, lons = [], []

mags = []

with open(filename) as f:

    # Create a csv reader object.

    reader = csv.reader(f)

    # Ignore the header row.

    #next(reader)

    # Store the latitudes and longitudes in the appropriate lists.

    for row in reader:

        lats.append(float(row[0]))

        lons.append(float(row[1]))

        mags.append(float(row[2]))

#----------------------------------------------------------------

from mpl_toolkits.basemap import Basemap

import matplotlib.pyplot as plt

import numpy as np

## Philippine Map coordinates

m = Basemap(resolution='f',projection='merc',                                        

            lat_0=13, lon_0=122,

            llcrnrlat=5.0,                                                         

            urcrnrlat=19.0,                                                          

            llcrnrlon=114.,                                                          

            urcrnrlon=130.0,                                                         

            ) 

m.drawmapboundary(fill_color='white')                                                

m.fillcontinents(color='#F5DEB3',lake_color='#85A6D9')                               

m.drawcoastlines(color='black', linewidth=.4)                                        

m.drawcountries(color='#6D5F47', linewidth=.4)                                       

min_marker_size= 2.5

for lon, lat, mag in zip(lons, lats, mags):

    x,y = m(lon, lat)

    msize = mag * min_marker_size 

    m.plot(x,y, 'ro', markersize=msize)

figsize=(50, 18)

plt.show()

DS.csv

09.97,124.17,2.6

11.64,126.10,3.2

11.55,126.31,3.1

11.59,126.23,4.9

04.87,127.19,3.7

06.17,126.02,2.5

14.79,120.00,2.3

05.69,126.26,4.2

14.74,119.91,5.9

Final image:

I'm Back.....

Wow, it's been years since I last blog. Since then, I finished my CCNP for Routing and Switching as well as for Security. I'm also been active CISSP since then.

Now for my biggest challenge of all..CCIE. I can say that this exam is a Beast. I've built my ccie home lab from old Cisco routers and switches and it's been a year since I started studying. The good thing about it is that the more I study, the more I know that there are lots of things to learn, and the the more things I've learn, the more I can design network efficiently and securely.

I'll be posting more configuration setup here for me to refresh my memory and hopefully help others as well.....

Happy Studying.....

Whoohaa

Finally got my CISSP exam result after a month of waiting, now time for endorsement. Also need 1 more exam for my CCNP..

iperf server

It's been a while since I last post. I've been busy designing and implementing our BGP routers and Core switches. Also, I've been studying for my CCNP and CISSP exams. (Whew!!).
Anyway, I just want to share the tool I used to test your network bandwidth (Internet, Site to site, etc.). I used iPerf in a linux box. It uses a client/server type of testing.

If you want an iPerf server to test your Internet link, send me your email so I can provide you the details...

Security Awareness 101

This is a good source of information for basic IT security awareness. It was used to educate people during APEC summit though it was old, the information is still relevant for today's technology.

http://aoema.org/SafetyNet/tabid/54/Default.aspx

How to start your Security Awareness Program?

I must say I'm still in the process of learning and practising IT Security but I know I got a good background in securing an environment as I worked on every aspect of Information Technology. In a usual IT environment, the company has Active directory for Identity and Access control, Firewall and router for Network Connectivity, Antivirus, Mail, Backup and File servers. Now how would you start your security program? You cannot just start mandating all your employees to do this and that. In able to have a successful security program, at least to start, is to have the Top executive back you up in the program. You must get their approval on why security is needed and how will the company benefit. The program should also align with the company's mission and vision statement. A program would be hard to implement if there is no budget allocated. In able to get the figures, you can start with Risk Assessment. This will give you an overall view on what your assets (Data, Information, Process, etc) are and which one needs to be secure. The assessment is a long process but it would be able to give you a figure on how much to secure the company's assets.

Now how would you direct the program to the end-user?. For me I started it by putting Posters at the common area. I printed on an A4 paper regarding how valuable a strong password are and how to create a strong and complex password.