MongoDb and M101J: MongoDB for Java Developers Answers

Welcome To Tripathi MongoDb Page

1.Introduction To MongoDb

MongoDB is a document database that provides high performance, high availability, and easy scalability.

* Document Database

o Documents (objects) map nicely to programming language data types.

o Embedded documents and arrays reduce need for joins.

o Dynamic schema makes polymorphism easier.

* High Performance

o Embedding makes reads and writes fast.

o Indexes can include keys from embedded documents and arrays.

o Optional streaming writes (no acknowledgments).

* High Availability

o Replicated servers with automatic master failover.

* Easy Scalability

o Automatic sharding distributes collection data across machines.

o Eventually-consistent reads can be distributed over replicated servers.

MongoDB supports MapReduce and other aggregation tools.

* Capped collections are fixed in size and are useful for certain types of data, such as logs.

* MongoDB supports an easy-to-use protocol for storing large files and file metadata.

* Some features common to relational databases are not present in MongoDB, notably joins and complex multirow transactions.

* MondgoDb provides master-slave replication

* If a master server goes down, MongoDB can automatically failover to a backup slave and promote the slave to a master.

* MongoDB’s administration philosophy is that the server should handle as much of the configuration as possible automatically.

2.MonoDb CRUD Operations:

* Insert Documents

* Query Documents

* Limit Fields to Return from a Query

* Iterate a Cursor in the mongo Shell

* Analyze Query Performance

* Modify Documents

* Remove Documents

* Perform Two Phase Commits

* Create Tailable Cursor

* Isolate Sequence of Operations

* Create an Auto-Incrementing Sequence Field

* Limit Number of Elements in an Array after an Update

2.1.Insert Documents Examples:

insert a Document with insert() Method¶

The following statement inserts a document with three fields into the collection inventory:

Ex1: db.inventory.insert( { _id: 10, type: "misc", item: "card", qty: 15 } )

2.db.products.insert( { item: "card", qty: 15 } )

3.

db.bios.insert(

{

name: { first: 'John', last: 'McCarthy' },

birth: new Date('Sep 04, 1927'),

death: new Date('Dec 24, 2011'),

contribs: [ 'Lisp', 'Artificial Intelligence', 'ALGOL' ],

awards: [

{

award: 'Turing Award',

year: 1971,

by: 'ACM'

},

{

award: 'Kyoto Prize',

year: 1988,

by: 'Inamori Foundation'

},

{

award: 'National Medal of Science',

year: 1990,

by: 'National Science Foundation'

}

]

}

)

The returned document includes an _id field with the generated ObjectId value:

{

"_id" : ObjectId("50a1880488d113a4ae94a94a"),

"name" : { "first" : "John", "last" : "McCarthy" },

"birth" : ISODate("1927-09-04T04:00:00Z"),

"death" : ISODate("2011-12-24T05:00:00Z"),

"contribs" : [ "Lisp", "Artificial Intelligence", "ALGOL" ],

"awards" : [

{

"award" : "Turing Award",

"year" : 1971,

"by" : "ACM"

},

{

"award" : "Kyoto Prize",

"year" :1988,s

"by" : "Inamori Foundation"

},

{

"award" : "National Medal of Science",

"year" : 1990,

"by" : "National Science Foundation"

}

]

}

Insert a Document Specifying an _id Field¶:

Ex:db.products.insert( { _id: 10, item: "box", qty: 20 } )

db.products.insert( { _id: 10, item: "box", qty: 50 } )

In the example, the document has a user-specified _id field value of 10. The value must be unique within the inventory collection.

Ex2:

db.bios.insert(

{

_id: 1,

name: { first: 'John', last: 'Backus' },

birth: new Date('Dec 03, 1924'),

death: new Date('Mar 17, 2007'),

contribs: [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ],

awards: [

{

award: 'W.W. McDowell Award',

year: 1967,

by: 'IEEE Computer Society'

},

{

award: 'National Medal of Science',

year: 1975,

by: 'National Science Foundation'

},

{

award: 'Turing Award',

year: 1977,

by: 'ACM'

},

{

award: 'Draper Prize',

year: 1993,

by: 'National Academy of Engineering'

}

]

}

)

db.bios.find( { _id: 1 } )

Insert Multiple Documents¶

db.bios.insert(

[

{

_id: 3,

name: { first: 'Grace', last: 'Hopper' },

title: 'Rear Admiral',

birth: new Date('Dec 09, 1906'),

death: new Date('Jan 01, 1992'),

contribs: [ 'UNIVAC', 'compiler', 'FLOW-MATIC', 'COBOL' ],

awards: [

{

award: 'Computer Sciences Man of the Year',

year: 1969,

by: 'Data Processing Management Association'

},

{

award: 'Distinguished Fellow',

year: 1973,

by: ' British Computer Society'

},

{

award: 'W. W. McDowell Award',

year: 1976,

by: 'IEEE Computer Society'

},

{

award: 'National Medal of Technology',

year: 1991,

by: 'United States'

}

]

},

{

_id: 4,

name: { first: 'Kristen', last: 'Nygaard' },

birth: new Date('Aug 27, 1926'),

death: new Date('Aug 10, 2002'),

contribs: [ 'OOP', 'Simula' ],

awards: [

{

award: 'Rosing Prize',

year: 1999,

by: 'Norwegian Data Association'

},

{

award: 'Turing Award',

year: 2001,

by: 'ACM'

},

{

award: 'IEEE John von Neumann Medal',

year: 2001,

by: 'IEEE'

}

]

},

{

_id: 5,

name: { first: 'Ole-Johan', last: 'Dahl' },

birth: new Date('Oct 12, 1931'),

death: new Date('Jun 29, 2002'),

contribs: [ 'OOP', 'Simula' ],

awards: [

{

award: 'Rosing Prize',

year: 1999,

by: 'Norwegian Data Association'

},

{

award: 'Turing Award',

year: 2001,

by: 'ACM'

},

{

award: 'IEEE John von Neumann Medal',

year: 2001,

by: 'IEEE'

}

]

}

]

)

2Query, Update and Projection Operators:

Query Selectors¶

Comparison¶

Name Description

$all Matches arrays that contain all elements specified in the query.

$gt Matches values that are greater than the value specified in the query.

$gte Matches values that are equal to or greater than the value specified in the query.

$in Matches any of the values that exist in an array specified in the query.

$lt Matches values that are less than the value specified in the query.

$lte Matches values that are less than or equal to the value specified in the query.

$ne Matches all values that are not equal to the value specified in the query.

$nin Matches values that do not exist in an array specified to the query.

Update Operators:

Fields¶

Name Description

$inc Increments the value of the field by the specified amount.

$rename Renames a field.

$setOnInsert Sets the value of a field upon documentation creation during an upsert. Has no effect on update operations that modify existing documents.

$set Sets the value of a field in an existing document.

$unset

Examples:

$all:

Syntax: { field: { $all: [ , ... ] }

db.products.insert( { _id: 20, item: "box", qty: 50 } )

db.products.insert( { _id: 10, item: "book", qty: 60} )

db.products.find({ item: { $all: ["box","book"]} })

db.inventory.find( { tags: { $all: [ "appliances", "school", "book" ] } } )

db.products.find( { qty: { $all: [ 20] } )

$gt:

db.products.find( { qty: { $gt: 25 } } )

$gte:

db.products.find( { qty: { $gte: 50 } } )

$in:

db.products.find( { qty: { $in: [ 50, 60 ] } } )

If the field holds an array, then the $in operator selects the documents whose field holds an array that contains at least one element that matches a value in the specified array (e.g. , , etc.)

$lt:

db.products.find( { qty: { $lt: 60 } } )

$ne:

db.products.find( { qty: { $ne: 20 } } )

$nin¶

Syntax: { field: { $nin: [ , ... ]} }

db.products.find( { qty: { $nin: [ 5, 15 ] } } )

3.Limit Fields to Return from a Query:

Return All Fields in Matching Documents:

If you specify no projection, the find() method returns all fields of all documents that match the query.

db.products.find( { item: 'book' } )

This operation will return all documents in the inventory collection where the value of the type field is 'food'. The returned documents contain all its fields.

Return Specified Fields Only:

You can remove the _id field from the results by specifying its exclusion in the projection, as in the following example:

db.products.find( { item: 'book' }, { qty: 50 } )

This operation returns all documents that match the query. In the result set, only the item and qty fields return in the matching documents.

4.Iterate a Cursor in the mongo Shell:

Manually Iterate the Cursor:

In the mongo shell, when you assign the cursor returned from the find() method to a variable using the var keyword, the cursor does not automatically iterate.

You can call the cursor variable in the shell to iterate up to 20 times [1] and print the matching documents, as in the following example:

var myCursor = db.products.find( { item: 'book' } );

myCursor

You can also use the cursor method next() to access the documents, as in the following example:

var myCursor = db.inventory.find( { type: 'food' } );

var myDocument = myCursor.hasNext() ? myCursor.next() : null;

if (myDocument) {

var myItem = myDocument.item;

print(tojson(myItem));

}

As an alternative print operation, consider the printjson() helper method to replace print(tojson()):

if (myDocument) {

var myItem = myDocument.item;

printjson(myItem);

}

Cursors :

-------

To create a cursor with the shell, put some documents into a collection, do a query on them, and assign the results to a local variable (variables defined with "var" are local). Here, we create a very simple collection and query it, storing the results in the cursor variable:

> for(i=0; i<100 i="" p="">

... db.c.insert({x : i});

... }

> var cursor = db.collection.find();

--> To iterate through the results, you can use the next method on the cursor. You can use hasNext to check whether there is another result. A typical loop through results looks like the following:

> while (cursor.hasNext()) {

... obj = cursor.next();

... // do stuff

... }

cursor.hasNext() checks that the next result exists, and cursor.next() fetches it.

--> The cursor class also implements the iterator interface, so you can use it in a forEach loop:

> var cursor = db.people.find();

> cursor.forEach(function(x) {

... print(x.name);

... });

adam

matt

zak

4.Analyze Query Performance:

db.products.find( { item: 'book' } ).explain()

5.Modify Documents:

Modify Multiple Documents with update() Method:

Refer http://docs.mongodb.org/manual/reference/sql-comparison/

6.Remove Documents:

Remove All Documents:

db.inventory.remove()

Remove Documents that Matches a Condition:

db.inventory.remove( { type : "food" } )

Remove a Single Document that Matches a Condition:

db.inventory.remove( { type : "food" }, 1 )

7.Perform Two Phase Commits:

Refer http://docs.mongodb.org/manual/tutorial/perform-two-phase-commits/

8.Create an Auto-Incrementing Sequence Field:

http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/

9.Limit Number of Elements in an Array after an Update:

http://docs.mongodb.org/manual/tutorial/limit-number-of-elements-in-updated-array/

db.patron.insert({

_id: "joe",

name: "Joe Bookreader"

})

db.address.insert({

patron_id: "joe",

street: "123 Fake Street",

city: "Faketon",

state: "MA",

zip: 12345

})

2.Data Models

Data Modeling Patterns

Model Embedded One-to-One Relationships Between Documents

Model Embedded One-to-Many Relationships Between Documents

Model Referenced One-to-Many Relationships Between Documents

Model Data for Atomic Operations

Model Tree Structures with Parent References

Model Tree Structures with Child References

Model Tree Structures with an Array of Ancestors

Model Tree Structures with Materialized Paths

Model Tree Structures with Nested Sets

Model Data to Support Keyword Search

Refer http://docs.mongodb.org/manual/data-modeling/ for Examples

Indexes

Indexes provide high performance read operations for frequently used queries. Indexes are particularly useful where the total size of the documents exceeds the amount of available RAM.

For basic concepts and options, see Indexing Overview. For procedures and operational concerns, see Indexing Operations. For information on how applications might use indexes, see Indexing Strategies.

Indexing Operations

Create an Index

Create a Compound Index

Create a Unique Index

Create a Sparse Index

Create a Hashed Index

Build Indexes on Replica Sets

Build Indexes in the Background

Remove Indexes

Rebuild Indexes

Monitor and Manage In Progress Index Creation and Building

Return a List of All Indexes

Measure Index Use

Build Old Style Indexes

For Examples:

http://docs.mongodb.org/manual/indexes/

Replication:

http://docs.mongodb.org/manual/core/replication-introduction/

MongoDbResources

http://www.mongodb.com/events

//Mangodb Basic Script Example

Install MongoDB on Ubuntu:

Step1:Configure Package Management System (APT)

root@thinkcloud-Desktop-STS:~# sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
[sudo] password for root:



Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
gpg: requesting key 7F0CEB10 from hkp server keyserver.ubuntu.com
gpg: key 7F0CEB10: public key "Richard Kreuter " imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)

Step2: Create a /etc/apt/sources.list.d/mongodb.list file using the following command.

root@thinkcloud-Desktop-STS:~# echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

Step3:Now issue the following command to reload your repository:

Install Packages:

Issue the following command to install the latest stable version of MongoDB:
root@thinkcloud-Desktop-STS:~#sudo apt-get update
Hit http://archive.canonical.com lucid Release.gpg
Ign http://archive.canonical.com/ubuntu/ lucid/partner Translation-en_IN
Hit http://ppa.launchpad.net lucid Release.gpg
Ign http://ppa.launchpad.net/ferramroberto/java/ubuntu/ lucid/main Translation-en_IN
Hit http://archive.canonical.com lucid Release
Hit http://security.ubuntu.com lucid-security Release.gpg
Ign http://security.ubuntu.com/ubuntu/ lucid-security/main Translation-en_IN
Ign http://security.ubuntu.com/ubuntu/ lucid-security/restricted Translation-en_IN
Ign http://security.ubuntu.com/ubuntu/ lucid-security/universe Translation-en_IN
Ign http://security.ubuntu.com/ubuntu/ lucid-security/multiverse Translation-en_IN
Hit http://ppa.launchpad.net lucid Release
Hit http://in.archive.ubuntu.com lucid Release.gpg
Ign http://in.archive.ubuntu.com/ubuntu/ lucid/main Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid/restricted Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid/universe Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid/multiverse Translation-en_IN
Hit http://in.archive.ubuntu.com lucid-updates Release.gpg
Ign http://in.archive.ubuntu.com/ubuntu/ lucid-updates/main Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid-updates/restricted Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid-updates/universe Translation-en_IN
Ign http://in.archive.ubuntu.com/ubuntu/ lucid-updates/multiverse Translation-en_IN
Get:1 http://downloads-distro.mongodb.org dist Release.gpg [490B]
Hit http://archive.canonical.com lucid/partner Packages
Hit http://ppa.launchpad.net lucid/main Packages
Hit http://security.ubuntu.com lucid-security Release
Hit http://in.archive.ubuntu.com lucid Release
Hit http://in.archive.ubuntu.com lucid-updates Release
Hit http://security.ubuntu.com lucid-security/main Packages
Ign http://downloads-distro.mongodb.org/repo/ubuntu-upstart/ dist/10gen Translation-en_IN
Hit http://in.archive.ubuntu.com lucid/main Packages
Hit http://in.archive.ubuntu.com lucid/restricted Packages
Hit http://in.archive.ubuntu.com lucid/main Sources
Hit http://in.archive.ubuntu.com lucid/restricted Sources
Hit http://in.archive.ubuntu.com lucid/universe Packages
Hit http://in.archive.ubuntu.com lucid/universe Sources
Hit http://security.ubuntu.com lucid-security/restricted Packages
Hit http://security.ubuntu.com lucid-security/main Sources
Hit http://security.ubuntu.com lucid-security/restricted Sources
Hit http://security.ubuntu.com lucid-security/universe Packages
Hit http://security.ubuntu.com lucid-security/universe Sources
Hit http://security.ubuntu.com lucid-security/multiverse Packages
Hit http://security.ubuntu.com lucid-security/multiverse Sources
Hit http://in.archive.ubuntu.com lucid/multiverse Packages
Hit http://in.archive.ubuntu.com lucid/multiverse Sources
Hit http://in.archive.ubuntu.com lucid-updates/main Packages
Hit http://in.archive.ubuntu.com lucid-updates/restricted Packages
Hit http://in.archive.ubuntu.com lucid-updates/main Sources
Hit http://in.archive.ubuntu.com lucid-updates/restricted Sources
Hit http://in.archive.ubuntu.com lucid-updates/universe Packages
Get:2 http://downloads-distro.mongodb.org dist Release [2,032B]
Hit http://in.archive.ubuntu.com lucid-updates/universe Sources
Hit http://in.archive.ubuntu.com lucid-updates/multiverse Packages
Hit http://in.archive.ubuntu.com lucid-updates/multiverse Sources
Ign http://downloads-distro.mongodb.org dist/10gen Packages
Ign http://downloads-distro.mongodb.org dist/10gen Packages
Get:3 http://downloads-distro.mongodb.org dist/10gen Packages [9,044B]
Fetched 11.6kB in 3s (3,610B/s)
Reading package lists... Done

Manage Installed Versions:

You can use the mongodb-10gen package to install previous versions of MongoDB. To install a specific release, append the version number to the package name, as in the following example:
root@thinkcloud-Desktop-STS:~#sudo apt-get install mongodb-10gen
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
mongodb-10gen
0 upgraded, 1 newly installed, 0 to remove and 319 not upgraded.
Need to get 87.1MB of archives.
After this operation, 221MB of additional disk space will be used.
Get:1 http://downloads-distro.mongodb.org/repo/ubuntu-upstart/ dist/10gen mongodb-10gen 2.4.8 [87.1MB]
Fetched 87.1MB in 2min 17s (634kB/s)
Selecting previously deselected package mongodb-10gen.
(Reading database ... 142828 files and directories currently installed.)
Unpacking mongodb-10gen (from .../mongodb-10gen_2.4.8_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Setting up mongodb-10gen (2.4.8) ...
Adding system user `mongodb' (UID 117) ...
Adding new user `mongodb' (UID 117) with group `nogroup' ...
Not creating home directory `/home/mongodb'.
Adding group `mongodb' (GID 124) ...
Done.
Adding user `mongodb' to group `mongodb' ...
Adding user mongodb to group mongodb
Done.
mongodb start/running, process 10298
To pin a package, issue the following command at the system prompt to pin the version of MongoDB at the currently installed version:
root@thinkcloud-Desktop-STS:~#echo "mongodb-10gen hold" | sudo dpkg –set-selections

Start MongoDB:

root@thinkcloud-Desktop-STS:~#sudo service mongodb start
start: Job is already running: mongodb

Stop MongoDB:

root@thinkcloud-Desktop-STS:~#sudo service mongodb stop

Restart MongoDB:

root@thinkcloud-Desktop-STS:~#sudo service mongodb restart
//Go to Mongo Command Promt:
root@thinkcloud-Desktop-STS:~# mongo
MongoDB shell version: 2.4.8
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:
Thu Jan 2 18:47:52.169 [initandlisten]
Thu Jan 2 18:47:52.169 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Thu Jan 2 18:47:52.169 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
Thu Jan 2 18:47:52.169 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
Thu Jan 2 18:47:52.169 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
Thu Jan 2 18:47:52.169 [initandlisten]
> db -To Check Default Db
test

//End of Installation Process

root@thinkcloud-Desktop-STS:~# sudo service mongodb start -To Start Mongodb

start: Job is already running: mongodb

root@thinkcloud-Desktop-STS:~# mongo

MongoDB shell version: 2.4.8

connecting to: test -default db

> db -to check current db name

test

> use test

switched to db test

> show dbs

local 0.03125GB

test (empty)

db.addUser( { user: "author", pwd: "pass", roles: [ "readWrite" ] } )

{

"user" : "author",

"pwd" : "9e67390049a2cb02f8f80ba8525233df",

"roles" : [

"readWrite"

],

"_id" : ObjectId("52c504ea53d2973d71745e85")

}

> show collections -To show all collections

system.indexes

system.users

> db.addUser( "guest", "pass", true )

{

"user" : "guest",

"readOnly" : true,

"pwd" : "c2a6b1fe136119b954d9e02eab627ab0",

"_id" : ObjectId("52c5059653d2973d71745e86")

}

> use test --To switched db est

switched to db test

//To Create Collection Tripathi and insert data

> db.tripathi.insert({fname:"shiva",lname:"tripathi",age:24})

//To see all Collections

> show collections

system.indexes

system.users

tripathi

//To Fecth Data from Collection

> db.tripathi.find()

{ "_id" : ObjectId("52c5068453d2973d71745e87"), "fname" : "shiva", "lname" : "tripathi", "age" : 24 }

//To Insert Data Into Collection

> db.tripathi.insert({fname:"kiran",lname:"nouli",age:27})

//To Fecth Data from Collection

> db.tripathi.find()

{ "_id" : ObjectId("52c5068453d2973d71745e87"), "fname" : "shiva", "lname" : "tripathi", "age" : 24 }

{ "_id" : ObjectId("52c5074e53d2973d71745e88"), "fname" : "kiran", "lname" : "nouli", "age" : 27 }

//To see All Collection

> show collections

system.indexes

system.users

tripathi

//To Fetch Only fname key data

> db.tripathi.find({},{fname:1,_id:0})

{ "fname" : "shiva" }

{ "fname" : "kiran" }

//To Create things as new colection and insert record into collection

> db.things.insert( { type : ['dog', 'cat'] } );

> db.things.insert( { egg : ['cat'] } );

> db.things.insert( { type : [] } );

> db.things.insert( { hello : [] } );

> db.things.insert( { hello : [] } );

> db.things.insert( { hello : [] } );

//To Fetch Data from things collection

> db.things.find()

{ "_id" : ObjectId("52c52adde7714ad11d724c50"), "type" : [ "dog", "cat" ] }

{ "_id" : ObjectId("52c52adde7714ad11d724c51"), "egg" : [ "cat" ] }

{ "_id" : ObjectId("52c52adde7714ad11d724c52"), "type" : [ ] }

{ "_id" : ObjectId("52c52ae2e7714ad11d724c53"), "hello" : [ ] }

{ "_id" : ObjectId("52c52ae8e7714ad11d724c54"), "hello" : [ ] }

{ "_id" : ObjectId("52c52aeae7714ad11d724c55"), "hello" : [ ] }

//To Fetch Data from things collection

> db.things.find({},{_id:0,type:1,egg:1,hello:1})

{ "type" : [ "dog", "cat" ] }

{ "egg" : [ "cat" ] }

{ "type" : [ ] }

{ "hello" : [ ] }

{ "hello" : [ ] }

{ "hello" : [ ] }

//Create new Collection shivanand and insert data into shivanand collection

> db.shivanand.insert({fname:"shiva"},{lname:"tripathi"},{age:24},{weight:54})

> ;

//Fetch Record from Colection

> db.shivanand.find()

{ "_id" : ObjectId("52c54c93e7714ad11d724c57"), "fname" : "shiva" }

//Fetch Record with particular key and id from Colection

> db.shivanand.find({},{fname:1})

{ "_id" : ObjectId("52c54c93e7714ad11d724c57"), "fname" : "shiva" }

//Fetch Record with particular key from Colection

> db.shivanand.find({},{fname:1,_id:0})

{ "fname" : "shiva" }

//Inser new Record into shivanand Collection

> db.shivanand.insert({fname:"kiran"},{lname:"nulpo"},{age:74},{weight:84})

> db.shivanand.insert({fname:"rajkumar"},{lname:"kampala"},{age:27},{weight:64})

> db.shivanand.insert({fname:"vamashi"},{lname:"chandraa"},{age:29},{weight:64})

//Fetch Record with particular key from Colection

> db.shivanand.find({},{fname:1,_id:0})

{ "fname" : "shiva" }

{ "fname" : "kiran" }

{ "fname" : "rajkumar" }

{ "fname" : "vamashi" }

//Sort _id in assending {_id:1}or dessending{_id:-1} order

//Script to find 2^rd and ^3rd record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(2).skip(2);

{ "_id" : ObjectId("52c54da8e7714ad11d724c59"), "fname" : "rajkumar" }

{ "_id" : ObjectId("52c54de5e7714ad11d724c5a"), "fname" : "vamashi" }

//Script to find ^1st and ^2nd record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(2).skip(1);

{ "_id" : ObjectId("52c54d73e7714ad11d724c58"), "fname" : "kiran" }

{ "_id" : ObjectId("52c54da8e7714ad11d724c59"), "fname" : "rajkumar" }

//Script to find 3rd record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(1).skip(3);

{ "_id" : ObjectId("52c54de5e7714ad11d724c5a"), "fname" : "vamashi" }

Note:For Single Rerod rank limit always 1

//Script to find 2^nd record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(1).skip(2);

{ "_id" : ObjectId("52c54da8e7714ad11d724c59"), "fname" : "rajkumar" }

//Script to find ^1st record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(1).skip(1);

{ "_id" : ObjectId("52c54d73e7714ad11d724c58"), "fname" : "kiran" }

//Script to find ^0th record indext count from 0

> db.shivanand.find().sort({_id:1}).limit(1).skip(0);

{ "_id" : ObjectId("52c54c93e7714ad11d724c57"), "fname" : "shiva" }

//Script to find ^0th record indext count from 0 and only key

> db.shivanand.find({},{fname:1,_id:0}).sort({_id:1}).limit(1).skip(0);

{ "fname" : "shiva" }

//Insert Recrod into shivanand collection

> db.shivanand.insert({fname:"rajbabau"},{lname:"omani"},{age:29},{weight:74})

> db.shivanand.insert({fname:"swapna"},{lname:"ponnam"},{age:26},{weight:54})

> db.shivanand.insert({fname:"anusha"},{lname:"bathi"},{age:24},{weight:54})

//Script to Find only fname key record

> db.shivanand.find({},{fname:1,_id:0})

{ "fname" : "shiva" }

{ "fname" : "kiran" }

{ "fname" : "rajkumar" }

{ "fname" : "vamashi" }

{ "fname" : "rajbabau" }

{ "fname" : "swapna" }

{ "fname" : "anusha" }

//Script to Find 5^th postion Record

> db.shivanand.find({},{fname:1,_id:0}).sort({_id:1}).limit(1).skip(5);

{ "fname" : "swapna" }

Q.What is the student_id of the lowest exam score above 65?

db.grades.find({type:'exam','score':{$gte:65}}).sort({'score':1},{limt:1}).pretty();
{
    "_id" : ObjectId("50906d7fa3c412bb040eb5cf"),
    "student_id" : 22,
    "type" : "exam",
    "score" : 65.02518811936324
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb743"),
    "student_id" : 115,
    "type" : "exam",
    "score" : 65.47329199925679
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb637"),
    "student_id" : 48,
    "type" : "exam",
    "score" : 65.71867938396781
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb65b"),
    "student_id" : 57,
    "type" : "exam",
    "score" : 65.91867871499709
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb6d3"),
    "student_id" : 87,
    "type" : "exam",
    "score" : 66.0470217410135
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb87f"),
    "student_id" : 194,
    "type" : "exam",
    "score" : 67.09136149008972
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb747"),
    "student_id" : 116,
    "type" : "exam",
    "score" : 67.09938431313856
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb733"),
    "student_id" : 111,
    "type" : "exam",
    "score" : 67.16752597563053
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb893"),
    "student_id" : 199,
    "type" : "exam",
    "score" : 67.33828604577803
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb6f7"),
    "student_id" : 96,
    "type" : "exam",
    "score" : 67.39154510277987
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb803"),
    "student_id" : 163,
    "type" : "exam",
    "score" : 67.51445811302555
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb683"),
    "student_id" : 67,
    "type" : "exam",
    "score" : 67.64289324359295
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb80f"),
    "student_id" : 166,
    "type" : "exam",
    "score" : 68.42859593432789
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb757"),
    "student_id" : 120,
    "type" : "exam",
    "score" : 68.695274154639
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb677"),
    "student_id" : 64,
    "type" : "exam",
    "score" : 71.07663195175958
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb86b"),
    "student_id" : 189,
    "type" : "exam",
    "score" : 71.2461033044598
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb717"),
    "student_id" : 104,
    "type" : "exam",
    "score" : 72.42031903025907
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb86f"),
    "student_id" : 190,
    "type" : "exam",
    "score" : 72.80738441829857
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb847"),
    "student_id" : 180,
    "type" : "exam",
    "score" : 72.82141551673207
}
{
    "_id" : ObjectId("50906d7fa3c412bb040eb857"),
    "student_id" : 184,
    "type" : "exam",
    "score" : 74.07556531186303
}
Type "it" for more
> db.grades.find({type:'exam','score':{$gte:65}}).sort({'score':1},{limt:1}).limit(1).pretty();
{
    "_id" : ObjectId("50906d7fa3c412bb040eb5cf"),
    "student_id" : 22,
    "type" : "exam",
    "score" : 65.02518811936324
}
> db.grades.find({type:'exam','score':{$gte:65}}).sort({'score':1}).limit(1).pretty();
{
    "_id" : ObjectId("50906d7fa3c412bb040eb5cf"),
    "student_id" : 22,
    "type" : "exam",
    "score" : 65.02518811936324
}
> db.grades.find({type:'exam','score':{$gte:65}},{_id:0,type:0,score:0}).sort({'score':1}).limit(1).pretty();

{ "student_id" : 22 }//Here $gte is Matches values that are equal to or greater than the value specified in the query.

> db.grades.find({type:'exam',score:{$lt:65}},{_id:0,type:0,score:0}).sort({score:1}).limit(1).pretty();
{ "student_id" : 191 }//Here $lt is Matches values that are less than the value specified in the query.

> db.grades.find({type:'exam',score:{$lte:65}},{_id:0,type:0,score:0}).sort({score:1}).limit(1).pretty();
{ "student_id" : 191 } //Here $lte is Matches values that are less than or equal to the value specified in the query.

> db.grades.find({type:'exam',score:{$gt:65}},{_id:0,type:0,score:0}).sort({score:1}).limit(1).pretty();
{ "student_id" : 22 } //Here $gt is Matches values that are greater than the value specified in the query.

> db.grades.find({type:'exam',score:{$ne:65}},{_id:0,type:0,score:0}).sort({score:1}).limit(1).pretty();
{ "student_id" : 191 } //Here $ne is Matches all values that are not equal to the value specified in the query.


$gt     Matches values that are greater than the value specified in the query.
$gte     Matches values that are equal to or greater than the value specified in the query.
$in     Matches any of the values that exist in an array specified in the query.
$lt     Matches values that are less than the value specified in the query.
$lte     Matches values that are less than or equal to the value specified in the query.
$ne     Matches all values that are not equal to the value specified in the query.
$nin     Matches values that do not exist in an array specified to the query.

M101J: MongoDB for Java Developers Answers:

1. Homework: Homework 1.1

   
   
   
   
   
   
   
   
   

   Install MongoDB on your computer and run it on the standard port. Download the HW1-1 from the Download Handout link and uncompress it.
   Use mongorestore to restore the dump into your running mongod. Do this by opening a terminal window (mac) or cmd window (windows) and navigating to the directory so that the dump directory is directly beneath you. Now type
   

   

    

   
   

Homework: Homework 1.2

Which of the following are valid JSON documents? Please choose all that apply.

Homework: Homework 1.3

Before starting this problem, you should have run mongorestore to set up your MongoDB database as described in HW1.1.

We want to test that you have a working JDK, that maven is installed and that you can run maven-based projects. Please install JDK 1.6 or above and maven if they are not already installed. This week's video lessons show how to install maven.

Download hw1-3.zip from Download Handout link, uncompress it, cd into the hw1-3 directory (there should be a pom.xml file in there), and run Maven as follows:

mvn compile exec:java -Dexec.mainClass=com.tengen.Week1Homework3

It requires Maven to be installed correctly, your mongod server to be running, and that you have run mongorestore properly. If it's working correctly, there should be two lines towards the bottom that say:

[INFO] -------------------
[INFO] BUILD SUCCESS

Homework: Homework 1.4

We are now going to test that you can run a Maven-based project that depends on the MongoDB Java driver, Spark, and Freemarker. Download hw1-4.zip, uncompress it, cd to the hw-1.4 directory, and run Maven as follows:

mvn compile exec:java -Dexec.mainClass=com.tengen.Week1Homework4

Like the previous homework, it requires Maven to be installed correctly, your mongod server to be running, and that you have run mongorestore properly from HW 1.1.
If all is working correctly, there should be two lines towards the bottom that say:

== Spark has ignited ...
>> Listening on 0.0.0.0:4567

Homework: Homework 2.1

In this problem, you will be using a collection of student scores that is similar to what we used in the lessons. Please download grades.js from the Download Handout link and import it into your local mongo database as follows: mongoimport -d students -c grades < grades.js The dataset contains 4 scores for 200 students. First, let’s confirm your data is intact; the number of documents should be 800.
> use students > db.grades.count()

 800 

Homework: Homework 2.2

Write a program in the language of your choice that will remove the grade of type "homework" with the lowest score for each student from the dataset that you imported in HW 2.1. Since each document is one grade, it should remove one document per student.
Hint/spoiler: If you select homework grade-documents, sort by student and then by score, you can iterate through and find the lowest score for each student by noticing a change in student id. As you notice that change of student_id, remove the document.
To confirm you are on the right track, here are some queries to run after you process the data with the correct answer shown:
Let us count the number of grades we have:

> db.grades.count() 
600

Now let us find the student who holds the 101st best grade across all grades:

> db.grades.find().sort({'score':-1}).skip(100).limit(1)
{ "_id" : ObjectId("50906d7fa3c412bb040eb709"), "student_id" : 100, "type" : "homework", "score" : 88.50425479139126 }

Now let us sort the students by student_id, score and see what the top five docs are:

> db.grades.find({},{'student_id':1, 'type':1, 'score':1, '_id':0}).sort({'student_id':1, 'score':1, }).limit(5)
{ "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
{ "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }
{ "student_id" : 1, "type" : "exam", "score" : 74.20010837299897 }

Quiz: Homework 2.3 (Python Manual Validation Version)

Preface to Homework 2.3 (Python Manual Validation Version)
There are two versions of homework 2.3. You only need to do one. This is the version that previous students have used. It involves running a python validation script on your local machine in order to verify that your blog is working, and so this version will include instructions on installing python. 

The blog stores its data in the blog database in two collections, users and sessions. Here are two example docs for a username ‘erlichson’ with password ‘fubar’. You can insert these if you like, but you don’t need to.

> db.users.find()
{ "_id" : "erlichson", "password" : "VH9IFu+/vUNSKTzZsFZEOsK1,-1924261330" }
> 
> db.sessions.find()
{ "_id" : "AN4M7warH+fdKOszU8qnd2Hmfn8JZFFZ9sff4zcPRpw=", "username" : "erlichson" }
> 

Ok, now it’s time to verify for yourself that you got it all working.

• go to http://localhost:8082/signup
• create a user

It should redirect you to the welcome page and say: welcome username, where username is the user you signed up with. Now

• Go to http://localhost:8082/logout
• Now go to http://localhost:8082/login.

If everything works, you should see your username when you are redirected to the welcome page.

There was one additional program that should have been downloaded in the project called validate.py. You can run it by typing

python validate.py

If you got it right, it will provide a validation code for you to enter into the box below. Enter just the code, in the answer box below, no spaces.

Notes on validation

The Python validation script requires the following python modules be installed on your computer: pymongo, urllib, urllib2, and cookielib. A typical Python installation will already have most of these installed except pymongo.  

Homework: Homework 3.1

Download the students.json file from the Download Handout link and import it into your local Mongo instance with this command:

$ mongoimport -d school -c students < students.json

This dataset holds the same type of data as last week's grade collection, but it's modeled differently. You might want to start by inspecting it in the Mongo shell.
Write a program in the language of your choice that will remove the lowest homework score for each student. Since there is a single document for each student containing an array of scores, you will need to update the scores array and remove the homework.
Hint/spoiler: With the new schema, this problem is a lot harder and that is sort of the point. One way is to find the lowest homework in code and then update the scores array with the low homework pruned. If you are struggling with the Python side of this, look at the remove operator, which can remove stuff from a Python list.
To confirm you are on the right track, here are some queries to run after you process the data with the correct answer shown:
Let us count the number of students we have:

> use school
> db.students.count() 
200

Let's see what Demarcus Audette's record looks like:

> db.students.find({_id:100}).pretty()
{
 "_id" : 100,
 "name" : "Demarcus Audette",
 "scores" : [
  {
   "score" : 47.42608580155614,
   "type" : "exam"
  },
  {
   "score" : 44.83416623719906,
   "type" : "quiz"
  },
  {
   "score" : 39.01726616178844,
   "type" : "homework"
  }
 ]
} 

Homework: Homework 3.2 (Python Manual Validation Version)

Preface to Homework 3.2 (Python Manual Validation Version)

There are two versions of homework 3.2. You only need to do one. This is the version that previous students have used. It involves running a python validation script on your local machine in order to verify that your blog is working, and so this version will include instructions on installing python.

You only need to do one version of homework 3.2. Do not do this version if you've already done the MongoProc version.

If you choose to do this version, you will need to download hw3-2and3-3.zip from the Download Handout link above, and unpack for the blog files, and also a validate.py file that we will use to verify on your local machine (port 8082) that your blog can accept posts.

Click this link to open a new tab to a YouTube playlist showing why you're installing python, and how to do it.

Making your blog accept posts
In this homework you will be enhancing the blog project to insert entries into the posts collection. After this, the blog will work. It will allow you to add blog posts with a title, body and tags and have it be added to the posts collection properly.

We have provided the code that creates users and allows you to login (the assignment from last week). To get started, please download hw3-2and3-3.zip from the Download Handout link and unpack. You will be using these file for this homework and for homework 3.3.

The areas where you need to add code are marked with "XXX". You need only touch the BlogPostDAO class. There are three locations for you to add code for this problem. Scan that file for XXX to see where to work.

Here is an example of valid blog post:

> db.posts.find().pretty()
{
    "_id" : ObjectId("513d396da0ee6e58987bae74"),
    "title" : "Martians to use MongoDB",
    "author" : "andrew",
    "body" : "Representatives from the planet Mars announced today that the planet would adopt MongoDB as a planetary standard. Head Martian Flipblip said that MongoDB was the perfect tool to store the diversity of life that exists on Mars.",
    "permalink" : "martians_to_use_mongodb",
    "tags" : [
        "martians",
        "seti",
        "nosql",
        "worlddomination"
    ],
    "comments" : [ ],
    "date" : ISODate("2013-03-11T01:54:53.692Z")
}

As a reminder, to run your blog you type:

mvn compile exec:java -Dexec.mainClass=course.BlogController

Or, use an IDE to run it.

To play with the blog you can navigate to the following URLs

http://localhost:8082/
http://localhost:8082/signup
http://localhost:8082/login
http://localhost:8082/newpost

You will be proving that it works by running our validation script as follows:

python validate.py

You need to run this in a separate terminal window while your blog is running and while the database is running. It makes connections to both to determine if your program works properly. Validate connects to localhost:8082 and expects that mongod is running on localhost on port 27017.

Read if you want to run your blog on non-standard port, different host or connected to a mongod on different server.

By popular demand, validate.py now takes some optional arguments that you can discover with the -h flag. These arguments will allow you to direct validate.py to make a connection to a web server at different port on a different host and connect to a mongod on a different host. You can also use a database name other than blog, a necessity if you

Homework: Homework 3.3 (Python Manual Validation Version)

Preface to Homework 3.3 (Python Manual Validation Version)

There are two versions of homework 3.3. You only need to do one. This is the version that previous students have used. It involves running a python validation script on your local machine in order to verify that your blog is working, and so this version will include instructions on installing python.

You only need to do one version of homework 3.3. Do not do this version if you've already done the MongoProc version.

If you choose to do this version, you will need to download hw3-2and3-3P.zip from the Download Handout link above, and unpack. This will include a validate.py file that we will use to verify on your local machine (port 8082) that your blog can accept posts.

Click this link to open a new tab to a YouTube playlist showing why you're installing python, and how to do it.

Making your blog accept posts
In this homework you will add code to your blog so that it accepts comments. You will be using the same code as you downloaded for HW 3.2.

Once again, the area where you need to work is marked with an XXX in the BlogPostsDAO class. There is only a single location you need to work to insert comments. You don't need to figure out how to retrieve comments for this homework because the code you did in 3.2 already pulls the entire blog post (unless you specifically projected to eliminate the comments) and we gave you the code in the template that pulls them out of the JSON document.

This assignment has fairly little code, but it's a little more subtle than the previous assignment because you are going to be manipulating an array within the Mongo document. For the sake of clarity, here is a document out of the posts collection from a working project that also has comments.

{
 "_id" : ObjectId("513d396da0ee6e58987bae74"),
 "author" : "andrew",
 "body" : "Representatives from the planet Mars announced today that the planet would adopt MongoDB as a planetary standard. Head Martian Flipblip said that MongoDB was the perfect tool to store the diversity of life that exists on Mars.",
 "comments" : [
  {
   "author" : "Larry Ellison",
   "body" : "While I am deeply disappointed that Mars won't be standardizing on a relational database, I understand their desire to adopt a more modern technology for the red planet.",
   "email" : "larry@oracle.com"
  },
  {
   "author" : "Salvatore Sanfilippo",
   "body" : "This make no sense to me. Redis would have worked fine."
  }
 ],
 "date" : ISODate("2013-03-11T01:54:53.692Z"),
 "permalink" : "martians_to_use_mongodb",
 "tags" : [
  "martians",
  "seti",
  "nosql",
  "worlddomination"
 ],
 "title" : "Martians to use MongoDB"
}

Note that you add comments in this blog from the blog post detail page, which appears at

http://localhost:8082/post/post_slug

where post_slug is the permalink. For the sake of eliminating doubt, the permalink for the example blog post above is http://localhost:8082/post/martians_to_use_mongodb
As a reminder, to run your blog you type:

mvn compile exec:java -Dexec.mainClass=course.BlogController

Or, use an IDE to run it.

To play with the blog you can navigate to the following URLs

http://localhost:8082/
http://localhost:8082/signup
http://localhost:8082/login
http://localhost:8082/newpost

You will run validate.py to check your work, much like the last problem. Validate.py will run through and check the requirements of HW 3.2 and then will check to make sure it can add blog comments, as required by this problem, HW 3.3. It checks the web output as well as the database documents. This validate.py program will print out a 3.3 validation code that you should enter below. Run our validation script as follows:

python validate.py

You need to run this in a separate terminal window while your blog is running and while the database is running. It makes connections to both to determine if your program works properly. Validate connects to localhost:8082 and expects that mongod is running on localhost on port 27017.

Read if you want to run your blog on non-standard port, different host or connected to a mongod on different server.

By popular demand, validate.py now takes some optional arguments that you can discover with the -h flag. These arguments will allow you to direct validate.py to make a connection to a web server at different port on a different host

Homework: Homework 4.1

Suppose you have a collection with the following indexes:

> db.products.getIndexes()
[
 {
  "v" : 1,
  "key" : {
   "_id" : 1
  },
  "ns" : "store.products",
  "name" : "_id_"
 },
 {
  "v" : 1,
  "key" : {
   "sku" : 1
  },
                "unique" : true,
  "ns" : "store.products",
  "name" : "sku_1"
 },
 {
  "v" : 1,
  "key" : {
   "price" : -1
  },
  "ns" : "store.products",
  "name" : "price_-1"
 },
 {
  "v" : 1,
  "key" : {
   "description" : 1
  },
  "ns" : "store.products",
  "name" : "description_1"
 },
 {
  "v" : 1,
  "key" : {
   "category" : 1,
   "brand" : 1
  },
  "ns" : "store.products",
  "name" : "category_1_brand_1"
 },
 {
  "v" : 1,
  "key" : {
   "reviews.author" : 1
  },
  "ns" : "store.products",
  "name" : "reviews.author_1"
 }

Homework: Homework 4.2

Suppose you have a collection called tweets whose documents contain information about the created_at time of the tweet and the user's followers_count at the time they issued the tweet. What can you infer from the following explain output?

db.tweets.find({"user.followers_count":{$gt:1000}}).sort({"created_at" : 1 }).limit(10).skip(5000).explain()
{
        "cursor" : "BtreeCursor created_at_-1 reverse",
        "isMultiKey" : false,
        "n" : 10,
        "nscannedObjects" : 46462,
        "nscanned" : 46462,

Homework: Homework 4.3 - Python Validation Version

Preface to Homework 4.3 (Python Manual Validation Version)

There are two versions of homework 4.3. You only need to do one. This is the version that previous students have used. It involves running a python validation script on your local machine in order to verify that your blog is working, and so this version will include instructions on installing python.

You only need to do one version of homework 4.3. Do not do this version if you've already done the MongoProc version.

If you choose to do this version, you will need to download hw4-3.zip from the Download Handout link above, and unpack. This will include a validate.py file that we will use to verify on your local machine (port 8082) that your blog can accept posts.

Click this link to open a new tab to a YouTube playlist showing why you're installing python, and how to do it.

Making the Blog fast Please download hw4-3.zip from the Download Handout link to get started. This assignment requires Mongo 2.2 or above.
In this homework assignment you will be adding some indexes to the post collection to make the blog fast.
We have provided the full code for the blog application and you don't need to make any changes, or even run the blog. But you can, for fun.
We are also providing a patriotic (if you are an American) data set for the blog. There are 1000 entries with lots of comments and tags. You must load this dataset to complete the problem.

# from the mongo shell
use blog
db.posts.drop()
# from the terminal window
mongoimport -d blog -c posts < posts.json

The blog has been enhanced so that it can also display the top 10 most recent posts by tag. There are hyperlinks from the post tags to the page that displays the 10 most recent blog entries for that tag. (run the blog and it will be obvious)
Your assignment is to make the following blog pages fast:

• The blog home page
• The page that displays blog posts by tag (http://localhost:8082/tag/whatever)
• The page that displays a blog entry by permalink (http://localhost:8082/post/permalink)

By fast, we mean that indexes should be in place to satisfy these queries such that we only need to scan the number of documents we are going to return. 

Homework: Homework 4.4 

Homework: Homework 5.1 (Hands On)

Finding the most frequent author of comments on your blog
In this assignment you will use the aggregation framework to find the most frequent author of comments on your blog. We will be using the same basic dataset as last week, with posts and comments shortened considerably, and with many fewer documents in the collection in order to streamline the operations of the Hands On web shell.

Use the aggregation framework in the web shell to calculate the author with the greatest number of comments.

To help you verify your work before submitting, the author with the fewest comments is Cody Strouth and he commented 68 times.

Once you've found the correct answer with your query, please choose your answer below for the most prolific comment author.

Note: this data set is relatively large. Due to some quirks of the shell, the entire result set gets pulled into the browser on find(), so if you want to see the document schema, we recommend either using db.posts.findOne(), db.posts.find().limit(1), or that you plan on waiting for a bit after you hit enter. We also recommend that the last phase of your aggregation pipeline is {$limit: 1} (or some single digit number)

>

Homework: Homework 5.2 (Hands On)

Crunching the Zipcode dataset
Please calculate the average population of cities in California (abbreviation CA) and New York (NY) (taken together) with populations over 25,000.

For this problem, assume that a city name that appears in more than one state represents two separate cities.

Please round the answer to a whole number.
Hint: The answer for CT and NJ (using this data set) is 38177.

Please note:

• One zip code may cover cities in different states.
• Different states might have the same city name.
• A city might have multiple zip codes.

Once you've generated your aggregation query, select your answer from the choices below.

For purposes of keeping the Hands On shell quick, we have used a subset of the data you previously used in zips.json, not the full set. This is why there are only 200 documents (and 200 zip codes), and all of them are in New York, Connecticut, New Jersey, and California.

 

Homework: Homework 5.3 (Hands On)

Who's the easiest grader on campus?
A set of grades are loaded into the grades collection.

The documents look like this:

{
 "_id" : ObjectId("50b59cd75bed76f46522c392"),
 "student_id" : 10,
 "class_id" : 5,
 "scores" : [
  {
   "type" : "exam",
   "score" : 69.17634380939022
  },
  {
   "type" : "quiz",
   "score" : 61.20182926719762
  },
  {
   "type" : "homework",
   "score" : 73.3293624199466
  },
  {
   "type" : "homework",
   "score" : 15.206314042622903
  },
  {
   "type" : "homework",
   "score" : 36.75297723087603
  },
  {
   "type" : "homework",
   "score" : 64.42913107330241
  }
 ]
}

There are documents for each student (student_id) across a variety of classes (class_id). Note that not all students in the same class have the same exact number of assessments. Some students have three homework assignments, etc.

Your task is to calculate the class with the best average student performance. This involves calculating an average for each student in each class of all non-quiz assessments and then averaging those numbers to get a class average. To be clear, each student's average includes only exams and homework grades. Don't include their quiz scores in the calculation.

What is the class_id which has the highest average student perfomance?

Hint/Strategy: You need to group twice to solve this problem. You must figure out the GPA that each student has achieved in a class and then average those numbers to get a class average. After that, you just need to sort. The hardest class is class_id=2. Those students achieved a class average of 37.6

Below, choose the class_id with the highest average student average.

 

Homework: Homework 5.4

Removing Rural Residents

In this problem you will calculate the number of people who live in a zip code in the US where the city starts with a digit. We will take that to mean they don't really live in a city. Once again, you will be using the zip code collection, which you will find in the 'handouts' link in this page. Import it into your mongod using the following command from the command line:

> mongoimport -d test -c zips --drop zips.json

If you imported it correctly, you can go to the test database in the mongo shell and conform that

> db.zips.count()

yields 29,467 documents.

The project operator can extract the first digit from any field. For example, to extract the first digit from the city field, you could write this query:

db.zips.aggregate([
    {$project: 
     {
 first_char: {$substr : ["$city",0,1]},
     }  
   }
])

Using the aggregation framework, calculate the sum total of people who are living in a zip code where the city starts with a digit. Choose the answer below.

Note that you will need to probably change your projection to send more info through than just that first character. Also, you will need a filtering step to get rid of all documents where the city does not start with a digital (0-9).

Homework: Homework 6.1:

Which of the following statements are true about MongoDB replication. Check all that apply.

Homework: Homework 6.2

Let's suppose you have a five member replica set and want to assure that writes are committed to the journal and are acknowledged by at least 3 nodes before you proceed forward. What would be the appropriate settings for w and j?

Homework: Homework 6.3

Which of the following statements are true about choosing and using a shard key:

Homework: Homework 6.4

You have a sharded system with three shards and have sharded the collections "grades" in the "test" database across those shards. The output of sh.status() when connected to mongos looks like this:

mongos> sh.status()
--- Sharding Status --- 
  sharding version: { "_id" : 1, "version" : 3 }
  shards:
 {  "_id" : "s0",  "host" : "s0/localhost:37017,localhost:37018,localhost:37019" }
 {  "_id" : "s1",  "host" : "s1/localhost:47017,localhost:47018,localhost:47019" }
 {  "_id" : "s2",  "host" : "s2/localhost:57017,localhost:57018,localhost:57019" }
  databases:
 {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
 {  "_id" : "test",  "partitioned" : true,  "primary" : "s0" }
  test.grades chunks:
    s1 4
    s0 4
    s2 4
   { "student_id" : { $minKey : 1 } } -->> { "student_id" : 0 } on : s1 Timestamp(12000, 0) 
   { "student_id" : 0 } -->> { "student_id" : 2640 } on : s0 Timestamp(11000, 1) 
   { "student_id" : 2640 } -->> { "student_id" : 91918 } on : s1 Timestamp(10000, 1) 
   { "student_id" : 91918 } -->> { "student_id" : 176201 } on : s0 Timestamp(4000, 2) 
   { "student_id" : 176201 } -->> { "student_id" : 256639 } on : s2 Timestamp(12000, 1) 
   { "student_id" : 256639 } -->> { "student_id" : 344351 } on : s2 Timestamp(6000, 2) 
   { "student_id" : 344351 } -->> { "student_id" : 424983 } on : s0 Timestamp(7000, 2) 
   { "student_id" : 424983 } -->> { "student_id" : 509266 } on : s1 Timestamp(8000, 2) 
   { "student_id" : 509266 } -->> { "student_id" : 596849 } on : s1 Timestamp(9000, 2) 
   { "student_id" : 596849 } -->> { "student_id" : 772260 } on : s0 Timestamp(10000, 2) 
   { "student_id" : 772260 } -->> { "student_id" : 945802 } on : s2 Timestamp(11000, 2) 
   { "student_id" : 945802 } -->> { "student_id" : { $maxKey : 1 } } on : s2 Timestamp(11000, 3) 

If you ran the query

use test
db.grades.find({'student_id':530289})

Which shards would be involved in answering the query?

Homework: Homework 6.5

In this homework you will build a small replica set on your own computer. We will check that it works with validate.py, which you should download from the Download Handout link.

Create three directories for the three mongod processes. On unix, this could be done as follows:

mkdir -p /data/rs1 /data/rs2 /data/rs3

Now start three mongo instances as follows. Note that are three commands. The browser is probably wrapping them visually.

mongod --replSet m101 --logpath "1.log" --dbpath /data/rs1 --port 27017 --smallfiles --oplogSize 64 --fork
 
mongod --replSet m101 --logpath "2.log" --dbpath /data/rs2 --port 27018 --smallfiles --oplogSize 64 --fork

mongod --replSet m101 --logpath "3.log" --dbpath /data/rs3 --port 27019 --smallfiles --oplogSize 64 --fork

Windows users: Omit -p from mkdir. Also omit --fork and use start mongod with Windows compatible paths (i.e. backslashes "\") for the --dbpath argument (e.g; C:\data\rs1).

Now connect to a mongo shell and make sure it comes up

mongo --port 27017

Now you will create the replica set. Type the following commands into the mongo shell:

config = { _id: "m101", members:[
          { _id : 0, host : "localhost:27017"},
          { _id : 1, host : "localhost:27018"},
          { _id : 2, host : "localhost:27019"} ]
};
rs.initiate(config);

At this point, the replica set should be coming up. You can type

rs.status()

to see the state of replication.

Now run validate.py to confirm that it works.

python validate.py

Validate connects to your local replica set and checks that it has three nodes. It has been tested under Pymongo 2.3 and 2.4. Type the validation code below.

MongoDB final exam:

Plese check your Question and Honor code Carefully Before using this Answer

Final: Question 1

Please download the Enron email dataset enron.zip, unzip it and then restore it using mongorestore. It should restore to a collection called “messages” in a database called “enron”. Note that this is an abbreviated version of the full corpus. There should be 120,477 documents after restore.

Inspect a few of the documents to get a basic understanding of the structure. Enron was an American corporation that engaged in a widespread accounting fraud and subsequently failed.

In this dataset, each document is an email message. Like all Email messages, there is one sender but there can be multiple recipients.

Construct a query to calculate the number of messages sent by Andrew Fastow, CFO, to Jeff Skilling, the president. Andrew Fastow’s email addess was andrew.fastow@enron.com. Jeff Skilling’s email was jeff.skilling@enron.com.

For reference, the number of email messages from Andrew Fastow to John Lavorato (john.lavorato@enron.com) was 1.

Option a. 1
Option b. 3
Option c. 5
Option d. 7
Option e. 9
Option f. 12

Answer Option b. 3

---

Final: Question 2

Please use the Enron dataset you imported for the previous problem. For this question you will use the aggregation framework to figure out pairs of people that tend to communicate a lot. To do this, you will need to unwind the To list for each message.

This problem is a little tricky because a recipient may appear more than once in the To list for a message. You will need to fix that in a stage of the aggregation before doing your grouping and counting of (sender, recipient) pairs.

Option a. susan.mara@enron.com to james.steffes@enron.com
Option b. susan.mara@enron.com to richard.shapiro@enron.com
Option c. soblander@carrfut.com to soblander@carrfut.com
Option d. susan.mara@enron.com to jeff.dasovich@enron.com
Option e. evelyn.metoyer@enron.com to kate.symes@enron.com
Option f. susan.mara@enron.com to alan.comnes@enron.com

Answer Option d. susan.mara@enron.com to jeff.dasovich@enron.com

---

Final: Question 3

In this problem you will update a document in the Enron dataset to illustrate your mastery of updating documents from the shell.

Please add the email address “mrpotatohead@10gen.com” to the list of addresses in the “headers.To” array for the document with “headers.Message-ID” of “<8147308 .1075851042335.javamail.evans="" thyme="">”

 db.messages.update(
{'headers.Message-ID': '<8147308 .1075851042335.javamail.evans="" thyme="">'},
{$addToSet: {
"headers.To": "mrpotatohead@mongodb.com"
}},
{multi: true}
)
 Click on Enter then right this answer and click on submit
Answer 897h6723ghf25gd87gh28

---

Final: Question 4

Enhancing the Blog to support viewers liking certain comments
In this problem, you will be enhancing the blog project to support users liking certain comments and the like counts showing up the in the permalink page.

Answer 983nf93ncafjn20fn10f

---

Final: Question 5

Suppose your have a collection fubar with the following indexes created:

[
 {
  "v" : 1,
  "key" : {
   "_id" : 1
  },
  "ns" : "test.fubar",
  "name" : "_id_"
 },
 {
  "v" : 1,
  "key" : {
   "a" : 1,
   "b" : 1
  },
  "ns" : "test.fubar",
  "name" : "a_1_b_1"
 },
 {
  "v" : 1,
  "key" : {
   "a" : 1,
   "c" : 1
  },
  "ns" : "test.fubar",
  "name" : "a_1_c_1"
 },
 {
  "v" : 1,
  "key" : {
   "c" : 1
  },
  "ns" : "test.fubar",
  "name" : "c_1"
 },
 {
  "v" : 1,
  "key" : {
   "a" : 1,
   "b" : 1,
   "c" : -1
  },
  "ns" : "test.fubar",
  "name" : "a_1_b_1_c_-1"
 }
]

Now suppose you want to run the following query against the collection.
db.fubar.find({‘a’:{‘$lt’:10000}, ‘b’:{‘$gt’: 5000}}, {‘a’:1, ‘c’:1}).sort({‘c’:-1})
Which of the following indexes could be used by MongoDB to assist in answering the query. Check all that apply.

Choice 1. _id_
Choice 2. a_1_b_1
Choice 3. a_1_c_1
Choice 4. c_1
Choice 5. a_1_b_1_c_-1

Answer
Choice 2. a_1_b_1
Choice 3. a_1_c_1
Choice 4. c_1
Choice 5. a_1_b_1_c_-1

---

Final: Question 6

Suppose you have a collection of students of the following form:

{
 "_id" : ObjectId("50c598f582094fb5f92efb96"),
 "first_name" : "John",
 "last_name" : "Doe",
 "date_of_admission" : ISODate("2010-02-21T05:00:00Z"),
 "residence_hall" : "Fairweather",
 "has_car" : true,
 "student_id" : "2348023902",
 "current_classes" : [
  "His343",
  "Math234",
  "Phy123",
  "Art232"
 ]
}

Now suppose that basic inserts into the collection, which only include the last name, first name and student_id, are too slow. What could potentially improve the speed of inserts. Check all that apply.

Choice 1: Add an index on last_name, first_name if one does not already exist.
Choice 2: Set w=0, j=0 on writes
Choice 3: Provide a hint to MongoDB that it should not use an index for the inserts
Choice 4: Remove all indexes from the collection
Choice 5: Build a replica set and insert data into the secondary nodes to free up the primary nodes.

Answer
Choice 2: Set w=0, j=0 on writes
Choice 4: Remove all indexes from the collection

---

Final: Question 7

You have been tasked to cleanup a photosharing database. The database consists of two collections, albums, and images. Every image is supposed to be in an album, but there are orphan images that appear in no album. Here are some example documents (not from the collections you will be downloading).

> db.albums.findOne()
{
“_id” : 67
“images” : [
4745,
7651,
15247,
17517,
17853,
20529,
22640,
27299,
27997,
32930,
35591,
48969,
52901,
57320,
96342,
99705
]
}

> db.images.findOne()
{ “_id” : 99705, “height” : 480, “width” : 640 }

From the above, you can conclude that the image with _id = 99705 is in album 67. It is not an orphan.

Your task is to write a program to remove every image from the images collection that appears in no album. Or put another way, if an image does not appear in at least one album, it’s an orphan and should be removed from the images collection.

Start by using mongoimport to import your albums.json and images.json collections. (Did you notice the links in the previous sentence?)
When you are done removing the orphan images from the collection, there should be 90,017 documents in the images collection. To prove you did it correctly, what are the total number of images with the tag ‘sunrises” after the removal of orphans? As as a sanity check, there are 50,054 images that are tagged ‘sunrises’ before you remove the images.
Hint: you might consider creating an index or two or your program will take a long time to run.

Option a. 43,434
Option b. 34,204
Option c. 49,123
Option d. 45,044
Option e. 50,054

Answer
Option d. 45,044

---

Final: Question 8

Supposed we executed the following Java code. How many animals will be inserted into the “animals” collection?

public class Question8 {
        public static void main(String[] args) throws IOException {
            MongoClient c =  new MongoClient(new MongoClientURI("mongodb://localhost"));
            DB db = c.getDB("test");
            DBCollection animals = db.getCollection("animals");


            BasicDBObject animal = new BasicDBObject("animal", "monkey");

            animals.insert(animal);
            animal.removeField("animal");
            animal.append("animal", "cat");
            animals.insert(animal);
            animal.removeField("animal");
            animal.append("animal", "lion");
            animals.insert(animal);

        }

}

Option a. 0
Option b. 1
Option c. 2
Option d. 3

Answer
Option b. 1

---

Final: Question 9

Imagine an electronic medical record database designed to hold the medical records of every individual in the United States. Because each person has more than 16MB of medical history and records, it’s not feasible to have a single document for every patient. Instead, there is a patient collection that contains basic information on each person and maps the person to a patient_id, and a record collection that contains one document for each test or procedure. One patient may have dozens or even hundreds of documents in the record collection.

We need to decide on a shard key to shard the record collection. What’s the best shard key for the record collection, provided that we are willing to run scatter gather operations to do research and run studies on various diseases and cohorts? That is, think mostly about the operational aspects of such a system.

Option a. patient_id
Option b. _id
Option c. primary care physican (your principal doctor)
Option d. date and time when medical record was created
Option e. patient first name
Option f. patient last name

Answer
Option a. patient_id

---

Final: Question 10

Understanding the output of explain We perform the following query on the enron dataset:
db.messages.find({‘headers.Date’:{‘$gt’: new Date(2001,3,1)}},{‘headers.From’:1, _id:0}).sort({‘headers.From’:1}).explain()
and get the following explain output.

{
 "cursor" : "BtreeCursor headers.From_1",
 "isMultiKey" : false,
 "n" : 83057,
 "nscannedObjects" : 120477,
 "nscanned" : 120477,
 "nscannedObjectsAllPlans" : 120581,
 "nscannedAllPlans" : 120581,
 "scanAndOrder" : false,
 "indexOnly" : false,
 "nYields" : 0,
 "nChunkSkips" : 0,
 "millis" : 250,
 "indexBounds" : {
  "headers.From" : [
   [
    {
     "$minElement" : 1
    },
    {
     "$maxElement" : 1
    }
   ]
  ]
 },
 "server" : "Andrews-iMac.local:27017"
}

Check below all the statements that are true about the way MongoDB handled this query.

Choice 1. The query did not utilize an index to figure out which documents match the find criteria.
Choice 2. The query used an index for the sorting phase.
Choice 3. The query returned 120,477 documents
Choice 4. The query performed a full collection scan

Answer
Choice 1. The query did not utilize an index to figure out which documents match the find criteria.
Choice 2. The query used an index for the sorting phase.
Choice 4. The query performed a full collection scan