DICOM Basics using Java - Query and Retrieve Operations (C-MOVE)
How the C-MOVE Composite Service Works
This article is part of my series of articles on the DICOM standard that I am currently working on (a number of them have already been completed). If you are totally new to DICOM, please have a quick look at my earlier article titled “Introduction to the DICOM Standard” for a quick introduction to the standard. It may also be useful to look at my other tutorials that have been completed so far to get up to speed on a number of topics including DICOM Encoding, SOPs and IODs. Before you dive into this article, please also have a look at my previous DICOM networking-related tutorials on DICOM Verification, on DICOM Associations and the DICOM C-FIND query operation as a basic understanding of those topics is required to understand the material that is covered here. My tutorial on DICOM C-STORE operation will also be useful to review since the "C-MOVE" operation actually uses the "C-STORE" as a 'sub-operation' under the covers to retrieve data. This tutorial also assumes that you know the basics of Java or any equivalent object-oriented language such as C# or C++. A basic understanding of networking will also be useful to have but is not mandatory.
Introduction
In our first tutorial on DICOM query and retrieval operations, we looked at how to find data on a remote location using the DICOM C-FIND query operation. In this tutorial, we are going to continue exploring the query/retrieval operations further by looking at first of the two DICOM operations that help us retrieve this data. I will cover the C-MOVE operation in this tutorial and will cover C-GET in a subsequent tutorial. We will build a DICOM C-MOVE SCU client that can instruct a remote DICOM server to either send back the DICOM data that we queried previously using C-FIND operation, or instruct the a remote server to move this data to another entirely different destination!
The PixelMed Java DICOM Toolkit - Quick Overview
For the purposes of illustrating many aspects of DICOM that I plan to cover in this tutorial series, I will be using a freely available and powerful DICOM toolkit called PixelMed Java DICOM Toolkit. This is a completely stand-alone DICOM toolkit that provides functionality for DICOM file and directory processing, image viewing as well as DICOM networking-related operations. This toolkit is completely free for both commercial or non-profit use. It is well documented and also has a small discussion forum and mailing list for users. The list of features contained within this toolkit is quite comprehensive. Please keep in mind that the use of this toolkit in my tutorial does not in anyway imply my official endorsement of it for implementing a production application. Every situation is unique, and only you are ultimately in the best position to decide that. This article is also not meant to be a tutorial on this toolkit, and my focus here is simply to tie DICOM theory to what a practical (although simple) implementation might look like. So, if your goal is to learn how to use the PixelMed library, I would encourage you to visit its website or check out the discussion forum or StackOverflow discussion pages for any assistance.
Before We Get Started…
Much like my previous programming examples, I will use the most bare minimum code and approach to help illustrate the concepts that I cover in this tutorial. This means that the code I write here is best suited to simply show the concept that I am trying to explain and is not necessarily the most efficient code to deploy in real life and in your production application.
To get started, you will need to configure a few things on your machine including a Java development environment as well as the PixelMed toolkit before you can run the example if you want to try this out yourself.
- Download and install the Eclipse Java IDE from here (or use any other IDE you prefer)
- Download the PixelMed toolkit library from here
- Ensure that the PixelMed.jar library is included in your Java project’s class path (some examples may require additonal runtime dependencies such as JAI Image IO Tools that can be found on PixelMed software download. Look for a tar compressed file called pixelmedjavadicom_dependencyrelease.YYYYMMDD.tar.bz2 or something similar)
- You can find the source code and images used in this tutorial on GitHub
- You can download more DICOM images from this site if you want as well
PACS Server Requirement
In addition to the tools described above, you will also need a DICOM server to execute some of the operations described in this tutorial. I will be using Orthanc Server that is installed and running locally on my machine for my examples shown below. There are many other open source PACS servers available on the Internet. However, I like using Orthanc Server since it is very easy to install and get started. Please see my article on getting started with Orthanc Server for more information. Please ensure that you have some studies uploaded already and can lookup patient studies within this data served by the Orthanc Server.
The other option is to use Dr. Dave Harvey's free online PACS server provided here. If you are going this route, please note that you will likely need to configure your firewall settings for any ports used on your machine (as well as your router if necessary) to allow the incoming traffic since the C-STORE SCU operation initiated by the remote server will attempt to reach the C-STORE SCP running on your machine through the specified ports. Dr. Dave Harvey is a radiologist by background who runs MedicalConnections, a software company which provides medical imaging-related technology consulting services for clients as well toolkits for developers in the DICOM space. Although I have never used his toolkit, I have reached out to him for help on DICOM matters in the past, and he was kind enough to point me to some very useful material to read or look at regarding DICOM. So, check out his toolkit if you are looking for a commercial solution for your DICOM requirement that is compatible with the Microsoft platform.
“One touch of nature makes the whole world kin” ~ William Shakespeare
Retrieving DICOM Data using C-MOVE
In my earlier tutorial on query operations in DICOM, we noted that the C-FIND service helps you find the DICOM objects you may be looking for. However, once you find what you are looking for, you now actually need to retrieve them. There are actually two composite operations that DICOM provides for this type of retrieval activity, namely C-MOVE and C-GET. Of these two operations, C-MOVE is the most popular method of retrieving DICOM files in clinical settings historically and will be the subject of this tutorial.
People who are new to DICOM will actually find the name of this operation a bit counter-intuitive since one is looking to retrieve data. However, the name makes sense since this operation can actually serve two purposes. It can not only retrieve data but also also direct a remote DICOM peer to send the data to a completely different destination which is something C-GET cannot do. I will cover the differences between the two operations in the tutorial on the C-GET composite operation. An interesting thing to note is that these activities are assisted behind the scenes through an entirely different composite operation called C-STORE which I covered in another article in this series. Behind the scenes, when the C-MOVE SCP receives a request, it actually initiates an entirely new DICOM association as a C-STORE SCU with the final destination and pushes the data to it. The final destination (the 'C-STORE-SCP') in most cases is the C-MOVE SCU that initiated the operation, but keep in mind that you can direct the store operation to push data to another destination like I mentioned earlier.
The C-STORE operation that happens under the covers is referred to as a 'sub-operation' in DICOM. The status of this sub-operation is communicated to the C-MOVE SCU to keep it informed of the overall progress. This C-STORE operation occurs over a completely different DICOM association with all the necessary association negotiation that will need to happen to ensure the abstract syntax and the transfer syntaxes are all acceptable to the receiver. Logically, an active C-STORE SCP service must be running on the destination for this overall operation to be successful. The diagram below hopefully captures the gist of what is really going during the C-MOVE operation. One thing to keep in mind is that the C-STORE SCU and the C-STORE SCP that communicate behind the scenes must know each other in advance (DICOM AE configurations must be in place on both sides to establish DICOM security). This is primarily because there is no 'dynamic authentication' in DICOM as seen in other communications protocols such as HTTP and any devices that communicate with one another using the DICOM protocol must be configured on both sides to know about one another.
From a DICOM communications perspective, a DIMSE command ('C-MOVE-RQ') coupled with a DICOM IOD object containing the criteria for the data that needs to be moved to a specified destination is transmitted from the client (C-MOVE-SCU) to the service provider (C-MOVE SCP). The criteria is specified through the "matching keys" attribute that we saw during our tutorial on C-FIND as well as through the use of another attribute called the query/retrieve level which specifies what we want to move (an entire study, series, image, etc). A character set may sometimes also be specified to control the character encoding of the value representations of the DICOM data that is returned. The C-MOVE SCP then switches roles to a C-STORE SCU and sends a 'C-STORE-RQ' command and any data matching the criteria to the specified destination. The C-MOVE SCP continues to communicate the status of the C-STORE operation such as the number of files transferred, number of files pending for transmission, any warnings and errors that arise during the operation, etc. The C-MOVE SCU may also initiate a cancel operation at anytime (using the 'C-CANCEL-MOVE-REQ' command) at which time any C-STORE operations that are in progress will be canceled, and a status of 'cancelled' in communicated back to the C-MOVE SCU. The C-MOVE request and response structures are shown below for reference. Please see the official DICOM documentation for more details as that is pretty all the theory I can cover in this article.
Example of C-MOVE Operation: Retrieving Data to the Caller
Let us take a look at a quick example using the PixelMed toolkit to perform the C-MOVE SCU operation. I am going to connect to an Orthanc Server running locally on my machine. I have configured the Orthanc Server to listen for connections on port 104. This server also has AE titles configured for both the C-FIND SCU client as well as the C-STORE SCP that it needs to push data back to. Again, Please see my article on getting started with Orthanc Server for more information. Again, as I mentioned previously,please ensure that you have some studies uploaded already and can lookup patient studies within this data served by the Orthanc Server..
In my example shown below, I am going to send a C-FIND request to find any studies belonging to a specific patient, and then create a C-MOVE request for any matched studies and have them sent them to my Java C-STORE SCP listening on port 11112. The code should be pretty self explanatory if you understood the theory behind this operation which was explained in the previous sections.
package com.saravanansubramanian.dicom.pixelmedtutorial;
import java.io.File;
import java.io.IOException;
import com.pixelmed.dicom.Attribute;
import com.pixelmed.dicom.AttributeList;
import com.pixelmed.dicom.AttributeTag;
import com.pixelmed.dicom.CodeStringAttribute;
import com.pixelmed.dicom.DicomException;
import com.pixelmed.dicom.SOPClass;
import com.pixelmed.dicom.SpecificCharacterSet;
import com.pixelmed.dicom.StoredFilePathStrategy;
import com.pixelmed.dicom.TagFromName;
import com.pixelmed.dicom.UniqueIdentifierAttribute;
import com.pixelmed.network.DicomNetworkException;
import com.pixelmed.network.FindSOPClassSCU;
import com.pixelmed.network.IdentifierHandler;
import com.pixelmed.network.MoveSOPClassSCU;
import com.pixelmed.network.ReceivedObjectHandler;
import com.pixelmed.network.StorageSOPClassSCPDispatcher;
public class DicomCMoveFunctionalityDemo {
public static void main(String arg[]) {
//Summary of what we are doing here:
//1. Start a C-STORE SCP server to be able listen for data pushed to us
//2. Perform a C-FIND operation for all studies matching a specific patient
//3. For each study found, retrieve all DICOM objects belonging to the study using a C-MOVE request
//3. As each file is received, write the information about the incoming data to the console
try {
String storeScpAeTitle = "JavaStoreScp";
int storeScpPortNumber = 11112;
File pathToStoreIncomingDicomFiles = new File("c:\\RetrievedDicomData");
//start a DICOM Store SCP Server to listen for data that will be pushed to us
Thread thread = new Thread(new StorageSOPClassSCPDispatcher(storeScpPortNumber, storeScpAeTitle, pathToStoreIncomingDicomFiles, StoredFilePathStrategy.BYSOPINSTANCEUIDINSINGLEFOLDER, new OurCMoveDemoStoreHandler()));
thread.start();
// use the default character set for VR encoding - override this as necessary
SpecificCharacterSet specificCharacterSet = new SpecificCharacterSet((String[])null);
AttributeList identifier = new AttributeList();
//build the attributes that you would like to retrieve as well as passing in any search criteria
identifier.putNewAttribute(TagFromName.QueryRetrieveLevel).addValue("STUDY"); //specific query root
identifier.putNewAttribute(TagFromName.PatientName,specificCharacterSet).addValue("ALLISON*");
identifier.putNewAttribute(TagFromName.PatientID,specificCharacterSet);
identifier.putNewAttribute(TagFromName.PatientBirthDate);
identifier.putNewAttribute(TagFromName.PatientSex);
identifier.putNewAttribute(TagFromName.StudyInstanceUID);
identifier.putNewAttribute(TagFromName.SOPInstanceUID);
identifier.putNewAttribute(TagFromName.StudyDescription);
identifier.putNewAttribute(TagFromName.StudyDate);
//retrieve all studies belonging to patient with name 'Bowen'
new FindSOPClassSCU("127.0.0.1",
104,
"ORTHANC",
"OurFindScu",
SOPClass.StudyRootQueryRetrieveInformationModelFind,
identifier,
new OurCMoveFindHandler());
}
catch (Exception e) {
e.printStackTrace(System.err); // in real life, do something about this exception
System.exit(0);
}
}
}
class OurCMoveFindHandler extends IdentifierHandler {
private static String moveSCPAddress = "127.0.0.1";
private static String moveScpAeTitle = "ORTHANC";
private static int moveScpPortNumber = 104;
private static String moveScuAeTitle = "JavaMoveScu";
private static String storeScpAeTitle = "JavaStoreScp";
@Override
public void doSomethingWithIdentifier(AttributeList attributeListForFindResult) throws DicomException {
System.out.println("Matched result:" + attributeListForFindResult);
String studyInstanceUID = attributeListForFindResult.get(TagFromName.StudyInstanceUID).getSingleStringValueOrEmptyString();
try {
AttributeList identifier = new AttributeList();
{ AttributeTag tag = TagFromName.QueryRetrieveLevel; Attribute attribute = new CodeStringAttribute(tag); attribute.addValue("STUDY"); identifier.put(tag,attribute); }
{ AttributeTag tag = TagFromName.StudyInstanceUID; Attribute attribute = new UniqueIdentifierAttribute(tag); attribute.addValue(studyInstanceUID); identifier.put(tag,attribute); }
new MoveSOPClassSCU(moveSCPAddress, moveScpPortNumber,moveScpAeTitle,moveScuAeTitle,storeScpAeTitle,SOPClass.StudyRootQueryRetrieveInformationModelMove,identifier);
}
catch (Exception e) {
System.out.println("Error during move operation" + e); // in real life, do something about this exception
e.printStackTrace(System.err);
}
}
}
class OurCMoveDemoStoreHandler extends ReceivedObjectHandler {
@Override
public void sendReceivedObjectIndication(String filename, String transferSyntax, String calledAetTitle)
throws DicomNetworkException, DicomException, IOException {
System.out.println("Incoming data from " + calledAetTitle + "...");
System.out.println("filename:" + filename);
System.out.println("transferSyntax:" + transferSyntax);
}
}
“Don’t walk in front of me, I may not follow. Don’t walk behind me, I may not lead. Walk beside me and be my friend” ~ Albert Camus
The console output below shows the operational log of all activities from my Java C-MOVE SCU client. You can the client establishing connection with the remote server, then retrieving the matched results and later persisting the retrieved DICOM objects on my local file system. Try passing in different criteria and explore the results returned as an additional exercise.
20:21:28,453 DEBUG [main] AssociationInitiator:330 - establishing Association
20:21:28,472 DEBUG [main] FindSOPClassSCU:330 - Association[0]: Hostname: 127.0.0.1
Association[0]: Port: 104
Association[0]: Called AE Title: ORTHANC
Association[0]: Calling AE Title: OurFindScu
[Presentation Context ID: 0x1 (result 0x0 - acceptance)
Abstract Syntax:
1.2.840.10008.5.1.4.1.2.2.1
Transfer Syntax(es):
1.2.840.10008.1.2.1
, Presentation Context ID: 0x3 (result 0x0 - acceptance)
Abstract Syntax:
1.2.840.10008.5.1.4.1.2.2.1
Transfer Syntax(es):
1.2.840.10008.1.2
, Presentation Context ID: 0x5 (result 0x0 - acceptance)
Abstract Syntax:
1.2.840.10008.5.1.4.1.2.2.1
Transfer Syntax(es):
1.2.840.10008.1.2.1
]
20:21:28,473 DEBUG [main] FindSOPClassSCU:347 - request identifier
(0x0008,0x0018) SOPInstanceUID VR=<UI> VL=<0x0> <>
(0x0008,0x0020) StudyDate VR=<DA> VL=<0x0> <>
(0x0008,0x0052) QueryRetrieveLevel VR=<CS> VL=<0x5> <STUDY>
(0x0008,0x1030) StudyDescription VR=<LO> VL=<0x0> <>
(0x0010,0x0010) PatientName VR=<PN> VL=<0x8> <ALLISON*>
(0x0010,0x0020) PatientID VR=<LO> VL=<0x0> <>
(0x0010,0x0030) PatientBirthDate VR=<DA> VL=<0x0> <>
(0x0010,0x0040) PatientSex VR=<CS> VL=<0x0> <>
(0x0020,0x000d) StudyInstanceUID VR=<UI> VL=<0x0> <>
20:21:28,474 DEBUG [main] FindSOPClassSCU:347 - Using context ID 5
20:21:28,478 DEBUG [main] FindSOPClassSCU:330 - waiting for PDUs
20:21:28,479 DEBUG [main] CompositeResponseHandler:330 - sendPDataIndication():
20:21:28,479 DEBUG [main] CompositeResponseHandler:330 - sendPDataIndication(): last fragment of command seen
20:21:28,482 DEBUG [main] FindSOPClassSCU:347 - evaluateStatusAndSetSuccess:
(0x0000,0x0000) CommandGroupLength VR=<UL> VL=<0x4> [0x4c]
(0x0000,0x0002) AffectedSOPClassUID VR=<UI> VL=<0x1c> <1.2.840.10008.5.1.4.1.2.2.1
Find result:(0x0008,0x0005) SpecificCharacterSet VR=<CS> VL=<0xa> <ISO_IR 100>
(0x0008,0x0018) SOPInstanceUID VR=<UI> VL=<0x36> <1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.205>
(0x0008,0x0020) StudyDate VR=<DA> VL=<0x8> <20150917>
(0x0008,0x0052) QueryRetrieveLevel VR=<CS> VL=<0x6> <STUDY >
(0x0008,0x1030) StudyDescription VR=<LO> VL=<0x2e> <TOMOSYNTHESIS SCREENING MAMMOGRAM W CAD G0202>
(0x0010,0x0010) PatientName VR=<PN> VL=<0xe> <ALLISON^MARY^A>
(0x0010,0x0020) PatientID VR=<LO> VL=<0xa> <2100016067>
(0x0010,0x0030) PatientBirthDate VR=<DA> VL=<0x8> <19511009>
(0x0010,0x0040) PatientSex VR=<CS> VL=<0x2> <O >
(0x0020,0x000d) StudyInstanceUID VR=<UI> VL=<0x36> <1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.206>
20:21:28,505 DEBUG [main] AssociationInitiator:330 - establishing Association
20:21:28,558 DEBUG [Thread-1] StorageSOPClassSCP:347 - Association received ORTHANC (127.0.0.1:49858) -> JavaStoreScp (127.0.0.1:11112)
Incoming data from ORTHANC...
filename:c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.209
transferSyntax:1.2.840.10008.1.2.1
20:21:29,062 DEBUG [Thread-1] StorageSOPClassSCP:347 - receiveAndProcessOneRequestMessage(): received file c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.209 from ORTHANC in 1.2.840.10008.1.2.1
Incoming data from ORTHANC...
filename:c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.208
transferSyntax:1.2.840.10008.1.2.1
20:21:29,137 DEBUG [Thread-1] StorageSOPClassSCP:347 - receiveAndProcessOneRequestMessage(): received file c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.208 from ORTHANC in 1.2.840.10008.1.2.1
Incoming data from ORTHANC...
filename:c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.205
transferSyntax:1.2.840.10008.1.2.1
20:21:29,208 DEBUG [Thread-1] StorageSOPClassSCP:347 - receiveAndProcessOneRequestMessage(): received file c:\RetrievedDicomData\1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.205 from ORTHANC in 1.2.840.10008.1.2.1
20:21:29,214 DEBUG [main] FindSOPClassSCU:347 - CFindResponseHandler.evaluateStatusAndSetSuccess: status = 0x0
20:21:29,214 DEBUG [main] FindSOPClassSCU:330 - CFindResponseHandler.evaluateStatusAndSetSuccess: status no longer pending, so stop
20:21:29,214 DEBUG [main] FindSOPClassSCU:330 - got PDU
20:21:29,215 DEBUG [main] FindSOPClassSCU:330 - releasing association
Example of C-MOVE Operation: Sending Retrieved Data to Another Destination
Handling the scenario is quite straight forward. Simply change the C-MOVE SCP destination that we specified in the earlier example to an entirely different DICOM peer. For easy testing, you should be able to use one of the many DICOM test utilities available on the Internet. The most popular ones in my opinion are those from DCMTK and DCM4CHE. Also, ensure that the Orthanc Server has an AE title which is configured correctly with the port that the test Store SCP server is going to listen on. Once this is all configured, simply run the Java program again and you should see the data pushed to the C-STORE SCP and the files should land in destination directory you specified. On my machine, I have got DCMTK's storescp test utility configured with a AE title of 'DCMTKSTORESCP', running on port 105 and directing any data received to be stored in the output directory of 'C:\RetrievedDicomData' on my machine. I also have the '-d' flag turned on to show the debug output on the console. This test utility provides a comprehensive dump of all DICOM communications that happen with the Orthanc Server. Note:I have only shown sections of the output that are useful to highlight the important events including association establishment as well as the storage of the incoming files pushed by the Orthanc Server which is acting as C-STORE SCP. The actual output displayed on the console will be much longer than is shown here.
C:\dcmtk-3.6.5-win64-dynamic\bin>storescp.exe 105 -aet DCMTKSTORESCP --output-directory C:\RetrievedDicomData -d
Association Received: DESKTOP-EIQC7KP
D: Parsing an A-ASSOCIATE PDU
I: Association Received
---
Other association establishment related debug information normally shown here
has been removed for sake of brevity....
----
I: Received Store Request
D: ===================== INCOMING DIMSE MESSAGE ====================
D: Message Type : C-STORE RQ
D: Presentation Context ID : 21
D: Message ID : 2
D: Affected SOP Class UID : DigitalMammographyXRayImageStorageForPresentation
D: Affected SOP Instance UID : 1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.208
D: Data Set : present
D: Priority : medium
D: Move Originator AE Title : JavaMoveScu
D: Move Originator ID : 2
D: ======================= END DIMSE MESSAGE =======================
I: storing DICOM file: C:\RetrievedDicomData\DXm.1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.208
D: DcmFileFormat::checkMetaHeaderValue() Version of MetaHeader is ok: 0x0001
D: DcmFileFormat::checkMetaHeaderValue() use SOPClassUID [1.2.840.10008.5.1.4.1.1.1.2] from Dataset
D: DcmFileFormat::checkMetaHeaderValue() use SOPInstanceUID [1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.208] from Dataset
D: DcmFileFormat::checkMetaHeaderValue() use new TransferSyntaxUID [Little Endian Implicit] on writing following Dataset
D: DcmFileFormat::validateMetaInfo() found 8 Elements in DcmMetaInfo 'metinf'
D: DcmDataset::read() TransferSyntax="Little Endian Implicit"
I: Received Store Request
D: ===================== INCOMING DIMSE MESSAGE ====================
D: Message Type : C-STORE RQ
D: Presentation Context ID : 21
D: Message ID : 3
D: Affected SOP Class UID : DigitalMammographyXRayImageStorageForPresentation
D: Affected SOP Instance UID : 1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.205
D: Data Set : present
D: Priority : medium
D: Move Originator AE Title : JavaMoveScu
D: Move Originator ID : 2
D: ======================= END DIMSE MESSAGE =======================
I: storing DICOM file: C:\RetrievedDicomData\DXm.1.2.276.0.7230010.3.1.4.651025538.23432.1587717786.205
I: Association Release
Conclusion
This concludes the article on how query and retrieval operations are enabled by the C-MOVE composite service in DICOM. This operation is used to retrieve DICOM data to the C-MOVE SCU or can be used send data to an entirely new destination. In the next tutorial in this series on the DICOM standard, I will cover another composite service called C-GET that also helps with data retrieval operations. I will also touch on the differences between C-GET operation and the C-MOVE operation and when and where it makes sense to use/deploy these two operations in the field. See you then!