eDJ Group

Analytics Adoption: Processing vs. Review

11 - 17 - 2014

Greg Buckles

One of the fun parts about publishing a new market perspective report is the hours of calls where I get to see what resonated with colleagues like Michael Simon and others. I don’t expect critical readers to agree with all of my conclusions. In fact, I enjoy the counterpoints as much as the congratulations after a long research cycle. One point keeps coming up over and over again that is worth extracting from my 26 page report. The report calls out pre-review processing and organization as the current market sweet spot for analytics, mainly because of the strong resistance to actual machine learning (PC/TAR) review methodologies reported in my expert interviews. So is there a real difference between a tech filtering collections based on clustered concepts or rules and an attorney training a predictive coding system? The market definitely sees them differently. Counsel rarely wrangle over exclusion of lunch notices, FaceBook messages or other non-relevant categories when they are even disclosed. Why not?

View Details

Categories:

Analytic Engine
Analytic Processing
Analytics
Technology Assisted Review

2014 Analytic Adoption Report - A Last

11 - 10 - 2014

Greg Buckles

After almost six months I have finally wrapped up my research report on analytic adoption in the eDiscovery and Information Governance markets. The report covers two surveys, twenty in depth interviews and focus briefings with eleven top technology providers. I started this research because I could not find any credible information that did not originate with or was sponsored by analytic providers. Most of my interviews went over an hour and revealed surprising resistance to reliance on machine-learning PC/TAR review systems.

View Details

Categories:

Analytic Engine
Analytic Processing
Analytics
Technology Assisted Review

Key Evidence vs. Discovery Collection Volumes

10 - 20 - 2014

Greg Buckles

Craig Ball raised a good point in his latest blog, Does the evidence we use derive from discovery?. Where he seems to be questioning the value of the entire discovery/eDiscovery process, I am more interested in getting some ad hoc metrics from you about what portion of preserved or collected paper and ESI could be considered actual ‘evidence’. My recollection from my earliest scanned paper productions and trials back in the 1990’s was that our exhibits always seemed to boil down to a single box (roughly 3,000 pages). It did not seem to matter how many boxes we scanned, reviewed and coded, the trial team always came up with a single box of printed exhibits to take to trial. Extensive depositions, expert analysis or third party productions would throw off this rule, but my impression was that the volume of key exhibits related more to the number of significant issues and the volume of documents that lead counsel was comfortable juggling in his/her head. I do remember a brilliant regulatory defense counsel from Baker Botts who had the ability to keep hundreds of trader email and voice conversations on instant recall as a good exception to the one box rule. So how does that ‘one box’ rule hold up 20+ years later?

View Details

Categories:

Collection Platform
Processing-ECA

Software Giants Repositioning Around IG Market

10 - 14 - 2014

Greg Buckles

My apologies for anyone who could not access the site last week. The legacy eDiscoveryJournal.com WordPress site was under attack and it took us quite a while to track down the resource drain. It was a big week for eDiscovery related M&A announcements. In brief; Microsoft is buying Equivio, Symantec is splitting, HP is splitting and Epiq facing a potential proxy fight. What’s next? Will IBM spin off Watson after pumping so much into it? You could say that MSFT is following their usual conservative go-to-market strategy by allowing smaller players innovate and take the risks of ‘jumping the maturity gap’. Symantec, HP and IBM all made eDiscovery acquisitions 2-4 years back. From my personal perspective, the acquired eDiscovery products (Clearwell, Autonomy and PSS Atlas/StoredIQ) all lost brand awareness and development momentum while being integrated into larger product portfolios. The Symantec/HP splits seem to buck the market consolidation trend until you look closer at the dividing lines drawn within each technology collective (everyone fears the Borg).

View Details

Categories:

EDD Services
Markets

A Look at Yammer Collection

09 - 30 - 2014

Greg Buckles

It is the middle of discovery interviews and you are going though your interview form checking off data sources with a key witness. You get to the final question, “Are there any other data sources, communication or collaboration systems that might be relevant to this matter?” You hold your breath, hoping that you can avoid collection of texts, Lync or other esoteric data sources that will blow up your processing and review budget estimates. “Well, the team started using Yammer back with Microsoft bought it. I think that we have a topic on that contract and I know that we have had several conversations about those pricing spreadsheets.” Ugh. Yammer? You take notes, get login credentials and the admins name. You have no idea what the system contains, how it is accessed or what you will get when you shoot the export request over to the Yammer administrator. Here is what it might look like…

View Details

Categories:

Office 365 Collection
Social Media Collection
Cloud ESI Collection

Why NOT Use PC-TAR?

09 - 22 - 2014

Greg Buckles

Mike Simon’s recent blogs really resonate with my take-aways from all of my interviews and survey results. The market is over-saturated with promises like “Review only 5,000 items!” and “Save 75% of review cost!” Although all of my cutting edge corporate eDiscovery interviewees had used some kind of machine learning in a few cases, none felt that PC-TAR was appropriate for ALL of matters. The dominant reason for using PC-TAR was to meet impossible deadlines or overwhelming volumes of raw ESI. PC-TAR was not considered easier or cheaper when you factored in the cost of the up front analytic processing ($130-200/GB extra), technical experts to manage the process and the cost to negotiate the protocol with the opposing party. So why did they go through all that trouble if they did not anticipate real savings? Because they did not have a choice. Many cited very tight time frames in recent regulatory requests. Others talked about being ‘dump trucked’ by huge opposing productions with bare weeks before depositions. My law firm interviews revealed a struggle between conservative partners and progressive litigation support evangelists. The eDiscovery providers gave me explicit descriptions of conversations with law firm partners with quotes like, “Your ROI for this stuff comes out of my pocket!” So here are my Top Ten Reasons Why NOT PC-TAR:

View Details

Categories:

Analytic Engine
Analytic Processing
Analytics
Technology Assisted Review

If This Is Cooperation, I Want To Go Back To Zealous Advocacy

09 - 16 - 2014

michael simon

A number of weeks ago, Greg Buckles posted the preliminary results of his survey on analytics usage in eDiscovery but then changed one of the key findings (as being too broadly asked) and instead drew a more specific, accurate conclusion from live interviews: My best estimation is that only 5-7% of matters that reach the review stage . . . actually use some form of PC-TAR. If you haven’t read my initial commentary on the survey, you can do so here. Reading my first article in this series will also go a long way towards explaining why I am using the term “machine learning” instead of “PC,” “PC-TAR,” “TAR,” “CAR/TAR,” “CAR,” or whatever. In short, calling machine learning by an accurate term could help with its adoption by cost-shocked corporate counsel and, at the very least, with existing adoption rates in the single digits, how could it hurt? Clearly, though, the disappointment of what was once the great hope of eDiscovery cost containment cannot be due solely to poor naming. The idea that a terrible name can hold back an otherwise great product is mostly modern mythology.

View Details

Categories:

Analytics
Technology Assisted Review

A Rose By Any Name

09 - 08 - 2014

michael simon

Why Aren’t Lawyers Using PC, or TAR, or PC-TAR, or CAR, or CAR-TAR, or Whatever We Call It? It’s been a long time—too long—since I have written anything for the eDJ Group. Once of my excuses is that I’ve been spending some time writing for some other sites, or put another way, exercising my inner snark. Several weeks ago, Greg Buckles posted preliminary results of his survey on analytics usage in eDiscovery. Greg then questioned his own results, or at least the raw statistics generated from his survey, presenting what he saw as the actual results: "My best estimation is that only 5-7% of matters that reach the review stage . . . actually use some form of PC-TAR." I’m not surprised at his general conclusion, though I do feel disappointed at those numbers. We all should feel that way; just a few years ago, machine learning technology represented something genuinely new and amazing. Even within the over-hyped, commercial blitz of the eDiscovery industry, predictive coding was genuinely promoted by many as the way that the ever-rising costs of eDiscovery, created by technology, could be solved by technology.

View Details

Categories:

Analytics
Technology Assisted Review

Non-business ESI Under Hold – BYOD Privacy vs. Preservation

09 - 04 - 2014

Greg Buckles

Management of traditional business information assets was relatively straight forward when the company owned and supplied everything but the phone service to employees. As we move into a cloud based, Bring Your Own Everything world of remote employees and mobile personal access, the conservative custodial legal hold approach throws such a wide net that it will certainly catch personal communications and other ESI. For smaller U.S. companies, this has driven up discovery costs without causing much concern over the inadvertent trampling of non-existent employee privacy rights. Global corporations are wrestling with segregating ESI from countries with serious civil and even criminal penalties for the same cavalier email harvesting, especially in the European Union. The June 25, 2014 decision by the Supreme Court of the United States (No. 13-132) soundly affirmed the expectation of privacy on mobile devices against law enforcement inspection without a search warrant. While this decision applies only to criminal ‘discovery’, a non-attorney interpretation can easily see potential impact for corporate discovery dragnets. Now that Microsoft’s new eDiscovery Center and most enterprise archives support ‘hold in place’ based on legal hold searches, custodians may lose the ability to ‘clean up’ their personal email and files that have accidentally been archived or migrated to SharePoint. So what can corporate legal and IT departments do when custodial privacy rights demand that their personal ESI be removed or excluded?

View Details

Categories:

Office 365 Collection
Social Media Collection
Enterprise Hold Platform
Legal Hold Notification

PC-TAR, Oversold and Under Adopted?

08 - 19 - 2014

Greg Buckles

Between my Analytic Adoption surveys and my 15+ extended interviews with cutting edge litsupport, attorneys and providers actively using analytics, the vast majority of consumers and cases are not yet ready for to use PC-TAR to make relevance decisions on productions. This goes against all everything that we have seen at Legal Tech and every other marketing channel in our industry. I am not questioning the potential effectiveness of the technology or the expertise of the subject matter experts that seem to be required to manage these machine learning reviews, whether active, passive or just plain black box driven. Instead, I asked my survey respondents to tell me about what percentage of cases they actively used analytics in. The initial numbers seemed quite encouraging (as seen below) until I started my live interviews. That was when I rediscovered the old axioms about Garbage In Garbage Out (GIGO) and Lies, Damn Lies and Statistics. Remember, this is my own data I am talking about.

View Details

Categories:

Analytics
Technology Assisted Review

Yammer eDiscovery?

08 - 11 - 2014

Greg Buckles

In 2012 Microsoft acquired Yammer, one of the top enterprise collaboration platforms. Think of it as business Facebook with document and conversation sharing functionality. Enterprise class Office 365 customers got the first Yammer integration back in November 2013 and Microsoft just extended Yammer to midsized business and education plans. A sad fact of innovation is that businesses can and will adopt new technology well before the corporate caretakers (Legal, Compliance & Security) have the tools to handle the new source. Last year, I spent a lot of time researching discovery of ESI from mobile devices from an enterprise perspective. The primary goal of many mobile discovery strategies is to minimize the unique ESI stored on those devices to make them essentially irrelevant to typical civil litigation. As much as I would love to use that same ‘duplicate source’ exclusion strategy on files uploaded to Yammer, the broad collaboration functions add critical context that will clearly be relevant to some matters if they are allowed to persist for anything beyond a few months. So have your users found Yammer and started using it with IT’s blessing or on their own? If you have opted for the enterprise system with limited administrative functions, have you had to preserve or collect from it?

View Details

Categories:

Office 365 Collection
Social Media Collection
Cloud ESI Collection
SharePoint Collection

Mergers, Acquisitions & eDiscovery

08 - 06 - 2014

Mikki Tomlinson

When it comes to M&A activity, the legal departments are typically last on the list of departments scheduled for merger planning. This is not unreasonable. “Business first” makes sense. It does, however, call for some stress and concern of potentially increased risk in the general counsel’s office(s). From the eDiscovery perspective, several questions arise as soon as a merger or acquisition is announced. Does the other organization have legal hold and discovery response procedures? What are their preservation practices? What eDiscovery technologies and relationships do they have? How different are their practices and technologies from ours? Are the legal departments going to merge or continue to run independently?

View Details

Categories:

eDJ 2014 Legal Hold Notification Survey

07 - 25 - 2014

Mikki Tomlinson

Case law related to failure to issue proper legal holds has not slowed over the last year. With that, and the uptick in legal hold notification products on the market over the last two years, we are seeing a rise in corporate attention to better managing legal hold notification. To test this postulation, we have issued our annual Legal Hold Notification Survey. In 2013 eDJ launched our first annual Legal Hold Notification Survey to understand corporate and government entity attitudes towards and plans for legal hold notification as part of their eDiscovery efforts. With the results of this year’s survey, we will study changes in trends over the last year.

View Details

Categories:

Legal Hold Notification

What Flavor of TAR is Right for You?

07 - 23 - 2014

Greg Buckles

My recent research and surveys have made it abundantly clear that ‘Analytics’ means many different things to the broad eDiscovery market. Even terms like Predictive Coding or Machine Learning can involve an infinite number of combinations of technologies, sampling strategies, training iterations and much more. Every provider seems to have a different take on how to leverage analytics in review. So what flavor of PC/TAR is right for you? For most consumers, the first priority is to differentiate between the major approaches that are offered as alternatives to traditional unorganized linear review. Here is my simplistic break down based on my active research, though I am excluding most pre-review culling methods.

View Details

Categories:

Technology Assisted Review

New Analytics Features in eDJ Matrix

07 - 14 - 2014

Greg Buckles

As I have been wrapping up my briefings with analytics providers and hearing them describe their ‘differentiating features’, I have been expanding the key features to support the new market categories for the eDJ Matrix. The surveys will run for another week or two, so if you want to get access to the direct results or the report, just log in and answer seven quick questions for Participating Member access when they are published. I have conducted roughly 15 focus interviews with active consumers and service providers. This is where you learn the really interesting bits. I have another week or so of briefings and demos of the latest technology releases. I will try to put out eDJ Briefs on these sessions, but expect them to be completely focused around user workflows and differentiating functions. There could be more analytic features if someone brings me a real game changer, but I feel pretty solid on the coverage when mapped against my new buying categories. That’s right. My intent is to only create categories that align with the way real people are consuming this technology, instead of the way that marketers seem to think that you should buy it.

View Details

Categories:

Technology Assisted Review
Processing-ECA

BYOD - Selling the Keys to the Corporate Kingdom?

07 - 10 - 2014

Greg Buckles

The overwhelming majority of corporations allow users to connect their personal smart phones to access work email and apps, called a Bring Your Own Device or BYOD policy. As I was publishing the results of my 2014 Mobile Discovery Survey, I ran across a rather inflammatory blog by security software company Avast! that detailed all the personal information retrieved from 20 Android phones that users had sold on eBay. I have heard stories from several security specialists concerning overseas hackers who set up web sites to troll for used phones from users with email addresses that match to company domains. I have never been able to corroborate those stories with any published stories or studies, but I long since updated client termination and BYOD policies and practices to make sure that outdated phones were wiped properly. The crux of Avast!’s bit of fear mongering is that a simple factory reset or even a remote ‘wipe’ will NOT actually delete or overwrite data on an Android OS phone. Ars Technica has a nice plain language explanation though some of their information was a bit out of date. The key point is that without encrypting the phones, the information is still there and can be recovered with forensic or even simple commercial software. Given that 21.4% of survey respondents (N=28) rely on users to manage mobile device security and access, this does raise a serious concern for corporations.

View Details

Categories:

Mobile Device Discovery

eDiscovery Analytics in the Trenches

06 - 24 - 2014

Greg Buckles

With over 50 survey responses and my first round of focus interviews under my belt, some trends are beginning to surface about how and why consumers and providers use advanced analytics. Early feedback and product briefings led me to launch separate consumer/provider surveys to better understand the different markets. The initial survey results have validated that decision, while I have tried to keep the 7 questions as close as possible to enable a merged market overview. The surveys will stay up for a couple more weeks while I go into another round of focus interviews. I target interviews for 15-30 minutes, but these keep running long with excellent insight from cutting edge practitioners. Right off the bat you can see below that providers and their customers have very different perceptions of the primary usage case for analytics.

View Details

Categories:

Technology Assisted Review

New Study Mired in the TAR Pit?

06 - 11 - 2014

Greg Buckles

The worlds of academic research and eDiscovery do not collide often enough. All too many practitioners assumed that first generation eDiscovery processing, search and collection technology were accurate and effective. I will stay off of my soap box on validation testing, but my long term readers know my passion for defensible, transparent process and tools. All too many ‘academic white papers’ in eDiscovery are funded from provider marketing budgets and even academic organizations such as the governmental Text Retrieval Conference (TREC) can have their results ‘reinterpreted’ by spin doctors to sell products to consumers desperate for reassurance. The hype cycle around Predictive Coding/Technology Assisted Review (PC/TAR) has focused around court acceptance and actual review cost savings. The last couple weeks have seen a bit of blogging kerfuffle over the conclusions, methods and implications of the new study by Gordon Cormack and Maura Grossman, “Evaluation of Machine-Learning Protocols for Technology-Assisted-Review in Electronic Discovery”. Pioneering analytics guru Herbert L. Roitblat of OrcaTec has published two blogs (first and second links) critical of the study and its conclusions. As much as I love a spirited debate and have my own history of ‘speaking truth’ in the public forum, I can’t help wondering if this tussle over Continuous Active Learning (CAL) vs. Simple Active Learning (SAL) has lost view of the forest while looking for the tallest tree in it.

View Details

Categories:

Technology Assisted Review

eDJ Brief: C2C Systems

06 - 05 - 2014

Greg Buckles

Overview: This briefing focused more on C2C’s strategic vision of ‘Pragmatic Information Management’ than my usual deep dive into a new release or technical foundations. If you have missed C2C, they provide enterprise software to manage Microsoft email, file and messaging capacity, discovery, retention and compliance needs of the mid market. They entered with email archiving market with ArchiveOne back in 2003. C2C’s value proposition is to enable customers to manage their unstructured data across live systems and archives from one interface. That now includes Office 365 email, but no mention of SharePoint or other non-email cloud data. They challenge the value of ‘integrated platforms’ for mid-market companies as being too expensive, complicated and difficult to implement. C2C seems on course to deliver non-disruptive tools for IT to tackle specific pain points such as migrations, defensible deletion rules and federated search/retrieval across unstructured data on premise and in the cloud.

View Details

Categories:

Archiving
Policy Management
Cloud ESI Collection

Back in the eDiscovery Saddle!

05 - 27 - 2014

Mikki Tomlinson

After a long journey over the last six months, I am now ready to get back in the eDiscovery saddle. Just after Thanksgiving 2013 I went on family medical leave. Coincidentally, eDJ Group also went through a lot of changes during that time. In this blog I will address both.

View Details

Categories: