Conventional methods for context based image retrieval include metadata-based and keyword-based approaches. Here, a simple and straightforward object based approach for retrieval of images is proposed. The features of the object under consideration, whose context need to be determined are stored beforehand in the system. When the query image containing the object is sent to the system, it detects the position of the object in the given image using Haar cascade classifier. Once the object is detected, the surrounding area of the object is scanned to summarize what kind of area the background is. The background information is extracted using color based segmentation. This information obtained, is used to formulate the context of the object. Unlike global retrieval methods, the proposed method focuses on a single application at a time.