IPython Notebook in pySpark mode on Ubuntu

Lately I have been very interested about Recommender Systems. According to 


I have started courses on Coursera and edX on the various types of Recommender system. All the courses makes use of a big data framework, (commonly Apache Spark over Hadoop).



This has necessitated learning about Apache Spark. Trying to run through a tutorial from codemento.io, The prerequisite was IPython notebook in pySpark mode
The means, Installing Apache Spark with IPython

• python
• ipython notebook
• Scala
• Java
• Apache Spark

sudo apt-get install default-jdk

sudo apt-get install ipython3 ipython3-notebook
or
pip install "ipython[notebook]"

wget http://www.scala-lang.org/files/archive/scala-2.11.7.deb
sudo dpkg -i scala-2.11.7.deb
sudo apt-get update
sudo apt-get install scala
sudo apt-get install -f

#prebuilt package of the latest Spark release, for Hadoop 2.4
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.2.0-bin-hadoop2.4.tgz
tar -xzvf spark-1.2.0-bin-hadoop2.4.tgz

cp spark-1.4.0-bin-hadoop2.6/ /opt/spark-1.4.0
cd /opt/spark-1.3.1
./bin/spark-shell
./bin/pyspark

MASTER="spark://127.0.0.1:7077" SPARK_EXECUTOR_MEMORY="6G" IPYTHON_OPTS="notebook --pylab inline" /opt/spark-1.3.1/bin/pyspark
MASTER="spark://127.0.0.1:7077" IPYTHON_OPTS="notebook --pylab inline" /opt/spark-1.3.1/bin/spark-shell

Windows PC as a Wireless Hotspot (2)

In the first post, we needed to install a third party software to convert the PC to a wireless hotspot. There are many hotspot software (free and paid) on the internet

While installing method is convenient, sometimes it might not be convenient to install one.
Windows have internal features that can be activated to convert your system into a Wireless router.

1. Open the Windows Command Prompt

2.  When it opens, type in the following command:

netsh wlan set hostednetwork mode=allow ssid=MyNet key=MyPassword

Replace 'MyNet' with a name you want to use for the custom network, and 'MyPassword' with a password that is hard to guess. Press enter to create it and then type in the following text:

netsh wlan start hostednetwork
Press enter again to start it up.

3. Lastly, go to Control Panel > Network and Internet > Network and Sharing Centre > Change Adapter Settings, right click your internet connection and select Properties. Select the Sharing tab and then check 'Allow other network users to connect…' and pick the virtual Wi-Fi adapter name you just created

4. Connect your wireless devices to the ssid used in Step 2

Moving On

It was been 3 tough months for me.

The Nigerian elections, and the related tension in the polity grounded the economy to a halt. Normally, during elections there seems to be influx of money into the economy.

2015 general elections seems not to have gotten the memo. If money was avaliable, it was not around me. I couldnt close any deal, and couldnt even revover from debtors.
Thanks to Dr Goodluck Jonathan concesssion phone call, the tension was sharply deflated.

In the midst of this madness, I lost my Dad. This was blow on two reasons. One, Losing a parent is traumatic, and then the accompaying burial arrangement is draining, mentally, physically, emotionally and lastly finanacially.

Considering the state of the nation ( and economy), the last took a lot of toll on us.

In spite of this, the we rode it out successfully. Thanks God for his mercies

Engr Ogbonna Uba Mgbeoji
1942-2015

Adieu Papa

#NigeriaDecides 28 March

After delays, and the propaganda and mudslinging, the election is taking places today.

Wishing for a violence free election.

Good Bless Nigeria

GE Lagos Garage Programmme

Today we completed a GE Garage Programmme in Lagos.

Wish all the participants.. a good luck in our life endeavors

Log in to ubuntu on virtualbox from host

We have been experimenting with different development environment in my startup.

python+virtualenv on Cygwin

I recently made Linux my platform for development, Firstly ue to the similarity with my production platform. Secondly because most open source programming tool performs optimally in Linux based system.

However, unless I replace my ageing system with a Mac, I will still need Windows OS for my small day to day works. So I have been doing most of my techical jobs on Ubuntu 12, installed ob Oracle VirtualBox.

However, I have been noticing some annoying delay, which prompted me to consider using Cygwin.

After pulling my hair a bit, I managed to figure out how to install python developement tools on Cygwin.

1. Install Python. 
2. Download and unzip pip. 
3. Install by going into the expanded directory and running python setup.py in a command prompt.
4. Set the %PYTHONHOME% system variable to the python base directory, (i.e. C:\Python27\) and adding the python base directory and script directory (i.e. C:\Python27\Scripts) to your %PATH% system variable.
5. Install Cygwin WITHOUT Python. The previous step tells Cygwin to use the Windows binary.
6. Install Cygwin-Virtualenvwrapper using pip install https://bitbucket.org/cliffxuan/virtualenvwrapper-for-cygwin-windows-python/get/tip.tar.gz
7. Install virtualenvwrapper-win using pip install virtualenvwrapper-win
8. Make a symlink between Cygwin's virtualenvhome directory and Windows's using ln -s /cygdrive/c/Users/<USER>/Envs/ ~/.virtualenvs
9. Add the following to Cygwin's .bashrc file: 

export VIRTUALENVWRAPPER_PYTHON="/cygdrive/c/Python27/python.exe"
export VIRTUALENVWRAPPER_VIRTUALENV="/cygdrive/c/Python27/Scripts/virtualenv.exe"
source virtualenvwrapper.sh

 
10. Go to C:\User\<username>\Env (or other %VIRTUALENV_HOME% location) and use virtualenv to start a new environment. Doing this allows virtualenvwrapper-win's workon command to work.