Multimedia Information Retrieval / Content-based Information Retrieval

Multimedia Information Retrieval

Definition:

Multimedia information retrieval (Multimedia IR) is a multimedia information system that can retrieve and store attributes, text, 2D grey-scale and color images, digitized audio or music and also video. It can be defined as the activity and process of providing users the information they are looking for.

Goals of Multimedia Information Retrieval:
۩Accuracy: Retrieve documents and information that the users expect to obtain
--> With few incorrect answers and information
۩Speed: Information retrievals has to be fast and can be obtained within seconds.




Published in all the multimedia information retrieval method is based on two principles :
1) Storage Oriented Retrieval Principle
The principle of the storage oriented principle is organization-oriented place where the documents are finally stored , or where reference to documents stored .


2) Probability Ranking Principle
If the reference retrieval system in response to the demands of each stage of documents in order to reduce the probability that the use of a collection of users who have
submitted a request , where the probability is estimated as accurately as possible on the basis of such data has been provided to the system for this purpose , the
overall effectiveness system to the consumer will be the best based on the data obtained.

Three Types of Multimedia Information Retrieval:

• Content-based Image Retrieval
• Content-based Video Retrieval
• Content-based Audio Retrieval

i) Content-based Image Retrieval

An application of searching for digital image in large databases.

ii) Content-based Video Retrieval

• Traditional video Retrieval such as Query-by-textual keyword.
• contain automatic visual concept detection. Example concepts are airplane, building , car, desert, explosion, outdoor, vehicle and violence.

iii) Content-based Audio Retrieval

An application to search sounds by features in the waveform, statistics, or transform domains.
--> eg: speech, music, silence and environment audio.




Methods of accessing – Text
۩ Multimedia documents consist of text (Captions are available for images)
۩ Text retrieval has several applications (for example: library automation and web searching )۩ Text retrieval researches have led to useful functions like information filtering and relevant feedback.




Approximate Searching
Information retrieval is not an exact process:۩ Two documents are not identical۩ and yet , they can be “similar” to each other.۩ Searching must be “approximately”




Error Counting:
۩ Retrieve similar or relevant words or phrases 
   - retrieve phrases related to misspelled words
۩ Editing distance : numerical estimate of the similarity between 2 strings
۩ Phonetic coding : search words or phrases with similar pronunciation 
۩ N-grams : count common N-length sub-strings 

The effectiveness of Information Retrieval depends on:
۩Types and precision of descriptions۩Types of queries allowed۩Efficiency of search techniques



Problems of Information Retrieval:
۩Descriptions of text for images and video found are not available or provided in some documents. For example, the text of a web site is not always descriptive of every image and video.
۩Two different approaches:
1)Human Annotations:
Online users add on attributes, captions for particular images and videos
→ retrievals will unsuccessful if queries are formulated using different keywords or descriptions
→ inconsistent (different users will give different descriptions and captions)
2)Feature extraction
Features are extracted from audio, image and video
-consistent descriptions but inexact
-difficult to extract meaning

Content-based Information Retrieval

What is content-based information retrieval?

Text document retrieval

۩ Text -based documents are basically unstructured and complex . They can only consists of raw text , tagged structures ( such as html documents ) , including
embedded images , and may have some fixed attributes that contain metadata that describes the aspects of the document .
۩ Web document describes the element of the semi-structured document . The original Grieg site also contains a list of references / links that provide access to
multimedia documents of composers , including some music .

How to improve multimedia information retrieval by using content-based methods .

When the text comment is nonexistent or incomplete in , the content-based approach is necessary. In addition, content-based method can potentially improve the retrieval accuracy .


Pratical application of CBIR
Crime preventation

• serious crime is commited,they can compare evidence from the scene crime for its similatary to records in their archivers.
• basic techniques for automatic fingerprint matching were worked out in the 1980s
• system based on this technology are how in routine use at the FBI in Washington
• Example – a numberof AFIS automatic fingerprint identification systems, AFIX,Tracker from Phoenix Comp-searching large database of fingerprint
• has been used for tall recognition also

Military

• Recognition of enemy aircraft from radan screw
• Identification of targets from satellite photograph
• Guidance system for cruise missiles
• Crime preventation

Intellectual property

• Copyright owners to seek and identity unauthorized copies of image
• Identification of illiat copies of image on the web

Architectural and Engineering Design

• Use of stylized 2D and 3D models
• Visualizing designs for the benefits of non-technical client
• Will meet specified suitability country

Fashion and Interior Designs

• Ability to search a collection of fabrics to find a particular combination of colour/texture is increasingly being recognize as a useful and to the design/process
• Colour matching of items
• Identify textile samples bearing a desired pattern

Journalism and advertising

• provide efficient d effective retrieval of still imagesfrom photo liratries
• For area of video asset management :
  CBIR needed to break up a video sequence into indiviual shots
  generate representative keyframes for each shot
• A degree of automatic video indexing and retrieval
  Mediakey Digital Video Library System from Islip Media
  Inc (based on Carnegie-Mellon University's highly-suceessful Information technology[Wactlar,1996]
  visionary from the Israel-based Media Acess Technologies Ltd(Wilf,1998)

Medical Diagnosis

• Identify medical images relating to a named patient
• CBIR works in PACS for a better quality of medical imaging

                            Content Based Image Information Retrieval
                                   

                            Content Based Video Information Retrieval


Reference:
http://www.intelligence.tuc.gr/~petrakis/courses/multimedia/retrieval.pdf
http://www.liacs.nl/~mlew/mir.survey16b.pdf
http://lyle.smu.edu/~mhd/8337sp09/mir.pdf
http://books.google.com.my/books?hl=en&lr&id=7ZhpFwcFxfwC&oi=fnd&pg=PA1&dq=multimedia+information+retrieval&ots=jSjJuwVFc9&sig=Vh8rTx_K807k1Dg5VTPfOkgxs_U#v=onepage&q&f=false
http://nordbotten.com/ADM/ADM_book/MIRS-frame.htm
http://www.eps.uam.es/esp/alumnos/trabajos_fin_master/Martinez_Martinez_Rafael.pdf
http://www.google.com.my/imgres?q=multimedia+information+retrieval&hl=en&sa=G&gbv=2&biw=1280&bih=699&tbm=isch&tbnid=6KtkqY7FtBe30M:&imgrefurl=http://www.science.uva.nl/research/publications/2008/LiICMIR2008/&docid=zNWq25Fc0kwkHM&w=6331&h=4363&ei=4OaJTsmAL8HOrQfBmtTvDA&zoom=1&iact=hc&vpx=380&vpy=145&dur=9212&hovh=186&hovw=271&tx=120&ty=82&page=1&tbnh=130&tbnw=189&start=0&ndsp=20&ved=1t:429,r:1,s:0
http://youtu.be/jy7B0knFfXA
http://www.google.com.my/imgres?q=content+based+information+retrieval&hl=en&gbv=2&biw=1280&bih=699&tbm=isch&tbnid=IkL_deNEB804vM:&imgrefurl=http://www.cs.auckland.ac.nz/compsci708s1c/lectures/Glect-html/topic1c708FSC.htm&do
http://www.google.com.my/imgres?q=content+based+information+retrieval&hl=en&gbv=2&biw=1280&bih=699&tbm=isch&tbnid=MF7gf-K90sPhXM:&imgrefurl=http://getglue.com/topics/p/co