Publication

Abstract : De-duplication is a common methodology which is been used for eliminating out the similar copies of same or repeating data in storage device of Cloud computing. A good majority of Cloud offer services processing of huge data. Although the resources are not limited in cloud, when the services are been expanded, there arises a high demand for storage space. The idea of data de-duplication gains its importance here by offering less storage space needs and hence saving the disk expenditures. However there do also exists some issues on data security and privacy. A number of methodologies are present for data security in cloud when using deduplication since the common deduplication techniques favors the deduplication algorithm to fetch and read data in each file. When coming into security challenges, the positives of deduplication unfortunately comes with high price. The proposed system assures deduplication in block-level and confidentiality of the data. Under the hood of Hadoop the system uses HDFS (Hadoop Distributed File System) for storing data. The same helps to store large files across multiple machines achieving reliability by replicating the data across multiple hosts. The overhead introduced by the mechanism is less and would not be impacting the cost for computation and overall storage.