Wednesday, June 13, 2012

Setting up Log4PHP with Code-Igniter

No BS, the steps are as following -

1. Clone the github project at https://github.com/fukata/ci-log4php

 

>: git clone https://github.com/fukata/ci-log4php.git

2. The ci-log4php directory has 2 folders. Copy and paste the ci_log4php folder in /application/third_party/

3. From the other folder Copy MY_Log.php file. Open config.php.
If $config['subclass_prefix'] = 'MY_';
then place the MY_Log.php file in /application/libraries/

4. Similarly Place the log4php.properties in /application/config folder and log4php_helper.php in /application/helpers folder

 

5. Edit the log4php.properties file. Set logs logs folder path

log4php.appender.default.file = /path/to/ci-app/application/logs/%s.log

6. Set the $config['log_threshold'] = 4; in config.php according to :

  • | 0 = Disables logging, Error logging TURNED OFF
  • | 1 = Error Messages (including PHP errors)
  • | 2 = Debug Messages
  • | 3 = Informational Messages
  • | 4 = All Messages

7. Go inside the application folder and run > chmod -R 777 ./logs

8. Use these commands for logging -

  // log_error('thiserror');

// log_info('thisinfo');

// log_debug('thisdebug');

Enabling SSL on MAC OS-X Snow Leopard

cd /private/etc/apache2/

openssl req -keyout privkey-$(date +%Y-%m).pem -newkey rsa:2048 -nodes -x509 -days 365 -out cert-$(date +%Y-%m).pem

Country Name (2 letter code) [AU]:CH State or Province Name (full name) [Some-State]:Zurich Locality Name (eg, city) []:Zurich Organization Name (eg, company) [Internet Widgits Pty Ltd]:Entropy Organizational Unit Name (eg, section) []:Secure Server Administration Common Name (eg, YOUR name) []:www.entropy.ch Email Address []:liyanage@access.ch

Make sure to enter the sitename properly.

Make sure that TextEdit is not running, then type these lines into the terminal window:

chmod 600 privkey-YYYY-MM.pem

chown root privkey-YYYY-MM.pem

open -a TextEdit /etc/apache2/httpd.conf

Uncomment the lines -

  • LoadModule ssl_module libexec/apache2/mod_ssl.so
  • Include /private/etc/apache2/extra/httpd-ssl.conf 

open -a TextEdit /etc/apache2/extra/httpd-ssl.conf

Edit these lines -

  • SSLCertificateFile /etc/apache2/cert-YYYY-MM.pem
  • SSLCertificateKeyFile /etc/apache2/privkey-YYYY-MM.pem

Restart apache

Media Coverage!

Symynd - Share your M'yn'd

Today I was moved to Symynd [http://www.symynd.com], a project built in Django, a web-development Framework built upon Python. Setting up the project on mac took quite some effort since I had to install a lot of Python and Django libraries to get the project running.

I am thinking of writing a script to make the setup process easy. Will probably work on that tomorrow.

Hadoop!

Hadoop is a large-scale distributed batch processing infrastructure. Batch processing is execution of a series of programs (jobs) on a computer without manual intervention. 

Hadoop includes a distributed file system which breaks up input data and sends fractions of the original data to several machines in your cluster to hold. This results in the problem being processed in parallel using all of the machines in the cluster and computes output results as efficiently as possible. 

Hadoop is designed to handle hardware failure and data congestion issues very robustly.

In a Hadoop cluster, data is distributed to all the nodes of the cluster as it is being loaded in. [Data is distributed across nodes at load time.]

The Hadoop Distributed File System (HDFS) will split large data files into chunks which are managed by different nodes in the cluster. In addition to this each chunk is replicated across several machines, so that a single machine failure does not result in any data being unavailable. Even though the file chunks are replicated and distributed across several machines, they form a single namespace, so their contents are universally accessible.

 Hadoop will not run just any program and distribute it across a cluster. Programs must be written to conform to a particular programming model, named "MapReduce."

Plan for the day

The plan for today is simple -

  • Read 2 chapters of TiJ
  • Make notes for the 2 chapters
  • Read Yahoo! tutorial on Hadoop
  • Optimize the codebase for the PlaceIQ project
  • Read an essay from Hackers and Painters
  • Solve some algorithm questions

Bloom Filters

Bloom filters are probabilistic data-structures built with the aim of handling specific usecases of huge DataSets while keeping the memory consumption minimum. Bloom filters are created by parsing the dataset once and once the entire dataset is parsed, the Bloom filters can be used to quickly Query if a particular Data item is there in the dataset or not.

An important thing to note is that Bloom filters can return False Positives but never False negatives. That means if a Query made on a Bloom Filter returns that an item doesnt exist in a dataset we can be sure about this result, but if the Query made on the Bloom Filter returns that data exists in the dataset, there is a small chance that the element might not exist in the dataset.

BloomFilters consist of a Number of hashtables of fixed sizes. Initially all the bits are set to zero in all the tables.

Each word of the dataset is hashed individually into the Bloom Filters using a single Hash Function or separate hash Functions, and the mod of the value returned by the hash is % with the size of the hashtable and the result key is set to 1.

The data-structure is probabilistic because there is a low but finite probability that two words will have collisions in each of the hash tables and Query might return true for one of them even though it might not be present in the original dataset. However, the chances of such errors are very low and BloomFilters can be customized, configured and optimized to better suit the given DataSet.

Many kinds of variations exist. One which use a single hash function for all tables has tables of varying lengths, so the % comes out to be different. Another implementation uses multiple hash functions but just a single hash table, where all the generated hashes are set.

From http://www.javamex.com/tutorials/collections/bloom_filter.shtml -

  • we allocate m bits to represent the set data;
  • we write a hash function that, instead of a single hash code, produces k hash codes for a given object;
  • to add an object to the set, we derive bit indexes from all k hash codes and set those bits;
  • to determine if an object is in the set, we again calculate the corresponding hash codes and bit indexes, and say that it is present if and only if all corresponding bits are set.

 

Java vs. C++

I ve been reading this book "Thinking in Java" and the Author keeps stressessing over the fact that Java is an improved version of C++ and other Object oriented languages in all aspects. The Java way to do things is always better and correct according to him.

Though he explains well why certain desicisions were taken by the developers of Java language, and how it benefits the developers, I would have appreciated the book more if he also talked about the flipside of the decisions taken while developing the language. Understanding the negative consequences of a design decision is as important as it is to know what benifits it provides.

UPDATE: On page 103, finally, the author says and I quote : 

... As you progress in this book, you ll see that many parts are simpler, and yet in other ways Java isn't much easier than C++ ...

 

Thinking in Java

Started reading the book Thinking in Java by Bruce Eckel. Aiming to complete the book in 3 weeks. Here is a list of chapters in the book -

Will strike out the names as i complete them.

  1. Introduction 13 May 5
  2. Introduction to Objects 2 May 5
  3. Everything Is an Object 61  May 5
  4. Operators 93 May 8
  5. Controlling Execution 135 May 8
  6. Initialization & Cleanup 155 May 8
  7. Access Control 209
  8. Reusing Classes 237
  9. Polymorphism 277
  10. Interfaces 311
  11. Inner Classes 345
  12. Holding Your Objects 389
  13. Error Handling with Exceptions 443
  14. Strings 503
  15. Type Information 553
  16. Generics 617
  17. Arrays 747
  18. Containers in Depth 791
  19. I/O 901
  20. Enumerated Types 1011
  21. Annotations 1059
  22. Concurrency 1109
  23. Graphical User Interfaces 1303

 

 

Getting started with Hadoop and MapReduce

Map Reduce is a programming paradigm developed for creating high scale data crunching programs by dividing the workload among several parallel machines. Hadoop MapReduce is the framework on which such programs are written.

Input data is fed as Key-Value pairs and the output is also in the form of Key-Value pairs, which enables Chaining of multiple MapReduce jobs one after the other.

This is what I ll be reading to get started -

http://developer.yahoo.com/hadoop/tutorial/

http://developer.yahoo.com/blogs/hadoop/

Ruby Java Bridge

A week ago while working on a Ruby on Rails project, I had to generate a highly complex excel file for certain Reporting requirements of the client. Previously I had used the Spreadsheet gem for generating the xls templates but it was clear spreadsheet was not going to be enough, since it works well on predefined templates only. We also looked at some other gems like WriteExcel but could not find a gem which was robust enough for our purpose.

When all hope was lost, we had to revert to Java, and fortunately Java had some jars which we could use for our requirement. We wrote the code in Java and using the Ruby Java Bridge gem, we could successfully generate the required excel report by reusing the Java code.

Link [Tutorial for RJB gem] http://www.ibm.com/developerworks/java/tutorials/j-rjb/index.html

The 6 URLs

This morning I got an email from my Manager instructing me to read and learn about Hadoop and MapReduce, along with six URLs to help me getting started -

  1. Map-Reduce Tutorial
  2. Apache Hadoop: Best Practices and Anti-Patterns
  3. Hadoop Performance Tuning
  4. 7 tips for improving Map-Reduce performance
  5. Migrating from Elastic MapReduce to a Cloudera's Distribution including Apache Hadoop Cluster
  6. Configuration paramaters - What can you just ignore?

Seems like an interesting project is coming up :)

TopCoder !

A friend talked about his idea of opening a startup today. His idea is to create a website similar to Pagalguy.com for engineering students (Pagalguy is for MBA). Agreed to develop the website for him if he could wait till July.

At Kuliza, I am researching over Groovy and Grails, a web development framework similar to ROR.

In other news, the awesome Topcoder tshirt finally arrived today; had lunch at Meghna Biryani; had a huge brawl with the house owner and the building watchman and finally decided to shift to a new home. Probably will shift to Bohmannahalli area. Travelling in Bangalore really sucks.

Dsc00806

Hello World!

++++++++++[>+++++++<-]>++.>++++++++++[>++++++++++<-]>+.>++++++++++[>++++++++++<-]>++++++++.>++++++++++[>++++++++++<-]>++++++++.>++++++++++[>+++++++++++<-]>+.>++++++++++[>+++<-]>++.>++++++++++[>++++++++<-]>+++++++.>++++++++++[>+++++++++++<-]>+.>++++++++++[>+++++++++++<-]>++++.>++++++++++[>++++++++++<-]>++++++++.>++++++++++[>++++++++++<-]>.>++++++++++[>+++<-]>+++.>

 

Thats Hello world! for you in BrainFuck Programming Language. To run the code, use the interpreter provided at http://brainfuck.tk/ :) 

Enabling SSL on MAC OS-X Snow Leopard

cd /private/etc/apache2/

openssl req -keyout privkey-$(date +%Y-%m).pem -newkey rsa:2048 -nodes -x509 -days 365 -out cert-$(date +%Y-%m).pem

Country Name (2 letter code) [AU]:CH State or Province Name (full name) [Some-State]:Zurich Locality Name (eg, city) []:Zurich Organization Name (eg, company) [Internet Widgits Pty Ltd]:Entropy Organizational Unit Name (eg, section) []:Secure Server Administration Common Name (eg, YOUR name) []:www.entropy.ch Email Address []:liyanage@access.ch

Make sure to enter the sitename properly.

Make sure that TextEdit is not running, then type these lines into the terminal window:

chmod 600 privkey-YYYY-MM.pem

chown root privkey-YYYY-MM.pem

open -a TextEdit /etc/apache2/httpd.conf

Uncomment the lines -

  • LoadModule ssl_module libexec/apache2/mod_ssl.so
  • Include /private/etc/apache2/extra/httpd-ssl.conf 

open -a TextEdit /etc/apache2/extra/httpd-ssl.conf

Edit these lines -

  • SSLCertificateFile /etc/apache2/cert-YYYY-MM.pem
  • SSLCertificateKeyFile /etc/apache2/privkey-YYYY-MM.pem

Restart apache

Setting up Log4PHP with Code-Igniter

No BS, the steps are as following -

1. Clone the github project at https://github.com/fukata/ci-log4php

 

>: git clone https://github.com/fukata/ci-log4php.git

2. The ci-log4php directory has 2 folders. Copy and paste the ci_log4php folder in /application/third_party/

3. From the other folder Copy MY_Log.php file. Open config.php.
If $config['subclass_prefix'] = 'MY_';
then place the MY_Log.php file in /application/libraries/

4. Similarly Place the log4php.properties in /application/config folder and log4php_helper.php in /application/helpers folder

 

5. Edit the log4php.properties file. Set logs logs folder path

log4php.appender.default.file = /path/to/ci-app/application/logs/%s.log

6. Set the $config['log_threshold'] = 4; in config.php according to :

  • | 0 = Disables logging, Error logging TURNED OFF
  • | 1 = Error Messages (including PHP errors)
  • | 2 = Debug Messages
  • | 3 = Informational Messages
  • | 4 = All Messages

7. Go inside the application folder and run > chmod -R 777 ./logs

8. Use these commands for logging -

  // log_error('thiserror');

// log_info('thisinfo');

// log_debug('thisdebug');

Wednesday, October 12, 2011

Is Schema.org the right way to go?

[ Originally written for Kuliza Technologies on June 14th, 2011 ]
Did the three big companies take the correct decision in introducing Schema.org ?

Semantic web is a web of information, which is marked with machine understandable metadata in addition to the Human readable web-content. Recently, Google, Yahoo and Microsoft collaborated and came up with Schema.org, which is their privately hosted Semantic mark-up vocabulary.
This introduction has been a hot topic of discussion in the Semantic web community, majorly because of the syntax chosen by the three companies to develop the vocabulary. The major issue with this release has been that the terms in Schema.org are expressed in microdata syntax, as opposed to the currently popular RDFa serialization of RDF. I am currently contributing open-source code to the Semantic web community through my project, which involves creating an RDF Vocabulary publishing platform. So maybe I might appear a bit biased towards RDFa over microdata here.

Bit of History -
RDF is a knowledge representation framework that encodes data as subject-predicate-object triples. When you combine triples, they form graphs. Initially, RDF/XML serialization format was used for semantic marking, and it separated the semantic marking from the HTML content. Over the course of time, Microformat syntax emerged, wherein the Semantic metadata content was integrated into the HTML itself. RDFa is another serialization of RDF, that was based on Microformat, i.e., integrating HTML Content and the metadata. Microdata is a set of tags, introduced with HTML5, which claimed to improve upon RDFa.
An important thing to note here is that RDFa and Microdata – both are syntaxes. Both are both Entity-Attribute-Value models that support using URIs as universal identifiers. There also exists an algorithm for converting Microdata to RDF. On the other hand, Schema.org is a vocabulary. A vocabulary has terms, which can be specified in any syntax. Schema.org terms have been originally specified in Microdata syntax.

Can’t we just specify all the terms in RDFa syntax and continue using them?
The answer is Yes, and as a matter of fact, the work is already in progress as I write this post. People in the RDFa community, Richard Cyganiak (My Google summer of code 2011 mentor) and Michael Hausenblas, have worked to develop an RDFS definition for the terms of Schema.org, and hosted it at http://schema.rdfs.org/.

So what is the issue here?
Google has asked the web community to use either microdata or RDFa since using both the syntaxes confuses its parsers.

“While it’s OK to use the new schema.org mark-up or continue to use existing Microformat or RDFa mark-up, you should avoid mixing the formats together on the same web page, as this can confuse our parsers.” … “If you have already done mark-up and it is already being used by Google, Microsoft, or Yahoo!, the mark-up format will continue to be supported. Changing to the new mark-up format could be helpful over time because you will be switching to a standard that is accepted across all three companies, but you don’t have to do it.”

And then it adds:
“We will also be monitoring the web for RDFa and Microformat adoption and if they pick up, we will look into supporting these syntaxes.”
This sounds as if Google is pushing developers who are looking for SEO to start using microdata syntax, a standard that is not in much use yet, since it gets a sort of priority in its parsing algorithms. This takes away the freedom from the developers to choose whatever syntax works best for them.  Although RDFa is a bit more complex than Microdata, it can covers more use cases, and some developers might be more comfortable using it.

Few years ago, the web-developers community was reluctant in semantically marking their web-content. The semantic web community worked hard to make the web developers understand the future benefits of having linked data all over the web. So, many of the developers slowly started using RDFa and Microformat, and a recent survey showed that 4% of websites used RDFa, which is more than any other. See http://tripletalk.files.wordpress.com/2011/01/rdfa-deployment.png for the comparison.

RDFa is being used by Drupal 7, Facebook OGP, Best Buy, all e-commerce sites which use the GoodRelations Vocabulary and many more major deployments globally.
And now schema.org asks them to learn a new syntax yet again. Lets face it; if Google, MS and yahoo declare that they would support only microdata for parsing content on the web, most of the web developers who are majorly looking for SEO would definitely follow. This would adversely affect the growth of RDFa deployments.

Thus, a large portion of the Semantic Web community is not happy with the decisions. Some believe that the vocabularies provided by schema.org won’t suffice if you want to cover complex domains since it is not extensible.

Another matter of concern is that it seems w3c was not consulted at all, while schema.org was developed. Commercialization of standards is never a good thing, and that’s what Schema.org does. In fact, Manu Sporny, chairperson of RDFa group in w3c, has been very aggressive in opposing schema.org and he goes to the extent of saying that he would soon start a revolution against “The false choice” of using microdata in schema.org. I have been following him on twitter and he has been gathering support there to put pressure on the three big Companies. He also believes that “Microdata doesn’t scale as easily as RDFa – early successes will be followed by stagnation and vocabulary lock-in.”

The solutions-
The most obvious solution to this problem is that Google, bing and yahoo announce that they would treat RDFa and microdata with equal priority in their parsing algorithms.
Bing has already stated that it can parse a page that includes multiple syntaxes. However, Google parsers cant do this, and needs to incorporate this feature in their parsing algorithms as soon as possible.

However…
Schema.org does seem to have a created a lot of negative buzz, but lets not forget that some kind of RDF vocabulary standardization like this was long due. Currently, due to lack of a definite standard, it is difficult for developers to decide on which one to use for mark-up. Schema.org does solve this problem and makes life easier for developers as well as for search engines. As Google states:

“Creating a schema supported by all the major search engines makes it easier for webmasters to add mark-up, which makes it easier for search engines to create rich search features for users.”

Friday, September 16, 2011

Kuliza@Mysore Day 1

We want to implelemt a working prototype of a sharing/analytics widget like easyshare, within 7 days. The widget is targetted at e-commerce platforms.


Problem :

Person 1 shares a link through our widget on facebook.
Person 2 shares the same link through our widget on facebook.
Person 3 reshares the same link on facebook by seeing Person 1's shared link.
Person 4,5 and 6 reshare the same link on facebook by seeing Person 2's shared link.
Person 7,8,9 and 10 see Person 3's link and reshare it.

We need to find who was the most influential person among each one who shared.


Tree models :
1 --- 3 --- (7,8,9,10)
2 --- (4,5,6)
As we can see here, Person 3 is the most influential here.


Solution :

Consider Person 1 and 2 as the root users. When a root user shares our link on facebook, we store the following in the backend :
1. Our system generated userid
2. Facebook id of the person
3. URL shared
4. parent Id : null

Then we append a query parameter to the URL which is the facebook User Id of the root user and shotren it using bit.ly, and then this link is shared.

eg. Person 1 shares http:www.hostname/productpage
we store Person 1's facebook userId and the URL http:www.hostname/productpage in the backend.

Now we append the url with a query parameter and shorten the url using bitly.

http:www.hostname/productpage/?q=userid
bit.ly/Li32df34

Then we share this link on facebook.

When a new person Re-shares the same link on facebook using the link provided by person 1:

Store the following in the database :
1. Our system generated userid
2. Facebook id of the new person
3. URL shared (got through the parent)
4. parent Id : id retrieved from the query parameter.

In this way, we can track the most influential user by querying to get the id which appears maximum time in the parentid.

Using bitly APIs, we can track the number of hits for a particular URL and hence find out the most influencial person.

Thursday, August 4, 2011

#Note PHP get contents of a remotely hosted file without cross-domain ajax

$file = file_get_contents('http://qa.agrinova.intuit.com/webmetrics/farmerCount.groovy');
echo $file;

Output :

<?xml version="1.0"?>
<webmetrics xmlns='http://agrinova.intuit.com'>
  <farmer_count>290158</farmer_count>
  <statewise_count state='GJ' count='135070' />
  <statewise_count state='AP' count='155088' />
</webmetrics>

 


Saturday, June 25, 2011

Updates : Porting Neologism to Drupal 7 [3]

A lot has happened in the project since the last post. In fact, I am ready with my mid-term submission.

The three content types : vocabulary, class and property have been added to the port. The fields that have been added are :

Vocabulary :
  • Title
  • Namespace URI
  • Authors
  • Abstract
  • Body
  • Additional Custom RDF

Class :
  • Related vocabulary
  • Class URI
  • Label
  • Comment
  • Superclass
  • Disjoint with 
  • Details

Property :
  • Related Vocabulary
  • Property URI
  • Label
  • Comment
  • Details
  • Functional Property
  • Inverse Functional Property
  • Domain
  • Range
  • Superproperty
  • Inverse
The vocabulary, class and property are correctly being registered with evoc.

Next steps are mentioned by Richard here-
http://drupal.org/node/1196510




Powered By Blogger
Custom Search